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Checklist  for  Designing  and  Implementing 
an  Evaluation 


[7]  Why  evaluate? 

Clearly  think  through  the  reason(s)  you  want  to  evaluate.  Consider  the 
evaluation  in  terms  of  your  program's  goals. 

What  do  I  need  to  know? 

Write  down  what  you  hope  to  learn. 

How  will  I  use  the  information? 

Before  you  begin  an  evaluation,  determine  how  you  will  use  the  findings. 
What  resources  are  available? 

Identify  available  resources  —  include  sources  of  technical  assistance. 

\7j  Are  my  research  questions  clear? 

Write  clear  and  specific  research  questions. 

Where  will  I  find  the  data  to  answer  research  questions? 

Identify  sources  of  information.  Data  availability  may  affect  your  choice 
of  a  particular  study  design. 

How  should  I  design  my  study? 

Choose  an  appropriate  evaluation  methodology,  one  that  suits  your 
research  questions  and  your  data  as  well  as  one  that  you  and  your  staff 
can  execute. 

What  does  data  collection  entail? 

Carefully  plan  for  data  collection.  Think  through  plans  to  identify  data 
sources,  design  data  collection  instruments,  train  data  collection  staff, 
and  collect  data.  Remember  to  use  existing  data;  it  requires  fewer 
resources. 

How  should  I  analyze  the  data? 

Analysis  and  assessment  are  inherently  judgmental.  First  calculate  simple 
measures  (such  as  frequencies  and  sample  means).  Apply  sophisticated 
statistical  techniques  only  if  your  data  will  support  them,  you  have  the 
skills  necessary  to  complete  such  analyses  (or  resources  to  obtain  help), 
and  the  findings  from  such  analyses  are  necessary  for  your  evaluation. 

What's  the  best  way  to  report  study  results? 

Think  about  the  best  way  to  get  your  study  findings  across.  Experiment 
with  tables,  charts,  and  graphs  that  reduce  narrative  and  provide 
easy-to-read  visual  aids. 
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Introduction 

Evaluation:  A  Vital  Management  Tool 


This  publication  is  a  practical  guide  for  WIC  administrators  to  design  and 
implement  program  evaluations.  The  emphasis  is  on  evaluation  of  critical 
issues  at  both  State  and  local  levels,  management  surveys,  and  assessment 
of  administrative  procedures  that  comprise  day-to-day  operations  in  local 
WIC  agencies. 

There  are,  of  course,  any  number  of  reasons  that  you  might  evaluate  some 
or  all  aspects  of  your  WIC  operations.  Such  information  can  be  used  for 
program  planning  and  budgeting;  for  policy  decisionmaking;  to  improve  or 
redesign  local  operations,  and  to  solve  problems  or  resolve  crises. 

Program  evaluation  can  involve  highly  complex,  multiyear,  multisite 
studies  that  require  numerous,  highly  trained  staff  and  significant  funding 
levels.  Or,  it  can  also  be  a  modest  descriptive  case  study  of  how  one 
program  delivers  a  particular  service.  You  need  to  decide  which  suits  your 
needs. 

Completing  multiyear  studies  is  clearly  an  important  task,  and  their  results 
help  guide  policy  and  program  decisions  at  national.  State,  and  local  levels. 
Of  equal  importance,  however,  are  shorter  term  administrative  studies  and 
management  surveys  that  focus  on  less  global  program  implementation 
issues.  Such  evaluations  examine  day-to-day  operations  in  local 
service-delivery  agencies.  They  focus  on  how  services  are  delivered,  the 
quality  of  service,  and  associated  costs. 

Seeking  Key  Answers 

A  primary  reason  why  WIC  managers  evaluate  their  operation  is  to 
obtain  information  that  is  administratively  useful.  Several  questions  set  the 
framework  for  this  type  of  evaluation: 

■  Are  these  activities  the  ones  we  should  be  carrying  out? 

■  Are  we  carrying  them  out  in  the  most  effective  and  efficient  fashion? 

■  By  completing  these  activities,  are  we  meeting  our  program  objectives? 

■  Is  our  program  effective? 

■  If  our  program  is  not  effective,  is  the  problem  individual  error,  special 
circumstances,  or  the  need  for  system  redesign? 

■  What  did  we  expect  to  see  as  results? 

■  What  is  different  than,  or  the  same  as,  what  we  expected? 

This  guide  provides  simple  approaches  to  answer  these  key  questions.  It 
offers  practical  guidance  on  evaluation.  It  also  includes  information  on 
evaluations  conducted  by  other  researchers  and  State  and  local  WIC 
agencies.  These  reports  can  serve  as  valuable  resources  as  you  face  issues 
and  resolve  problems  in  your  agency. 
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/.  Defining  Evaluation 


For  the  public  sector,  an  evaluation  is  a  careful 
consideration  of  the  objectives,  operations,  and  effects 
of  a  publicly  supported  program  or  activity. 
Evaluation  can  measure  program  performance  and 
often  involves  comparing  one  set  of  program 
measures  with  another  set  of  measures  —  either  of  the 
same  program,  of  other  programs,  or  with  a  more 
general  set  of  comparable  measures.  Program 
performance  includes  assessing  program  activities  as 
well  as  identifying  the  outcomes  of  these  activities. 
Measuring  "performance"  means: 

■  considering  progress  toward  meeting 
program  objectives 

■  estimating  the  use  of  resources 

■  gauging  effects  on  persons  served  by 
the  program 

Evaluation  may  also  include  an  assessment  of  the 
environment  in  which  a  program  operates,  as  well  as 
the  program's  effects  on  that  environment. 

Types  of  Evaluations 

There  are  several  types  of  evaluation.  Program 
monitoring  activities,  feasibility  studies,  and  outcome 
studies  are  all  different  types  of  evaluations.  You  will 
find  that  studies  may  have  characteristics  of  different 
types  of  evaluations.  Three  types  of  evaluations  often 
used  to  assess  WIC  as  well  as  other  social  programs 
are: 

■  Process  or  implementation  studies  —  descriptive 
evaluations  that  provide  detailed  information  on  the 
execution  of  an  activity  or  series  of  activities.  For 
example,  all  WIC  local  agencies  are  required  to 
provide  nutrition  education  services  to  their  clients, 
so  you  might  want  to  know  how  this  aspect  of  WIC  is 
being  met.  Questions  include:  How  does  the  agency 
provide  nutrition  education  (group  sessions, 
one-on-one  counseling)?  What  are  the  day-to-day 
operations?  What  is  the  average  length  of  time  for 
nutrition  education  contacts?  Who  does  them?  How 
much  does  this  service  cost? 

■  Impact  analyses  or  outcome  evaluations  —  usually 
quantitative  evaluations  that  measure  the  outcome  of 
an  activity  or  program.  They  may  also  focus  on 
whether  or  not  there  is  a  "problem"  that  the  program 


solves  by  providing  certain  services.  To  use  the 
nutrition  education  example  again,  an  impact  analysis 
would  ask  such  questions  as:  What  are  the  effects  on 
WIC  clients?  How  many  clients  attend  group  sessions? 
What  were  the  effects  of  these  services  on  clients  with 
regard  to  changes  in  dietary  habits  and /or  nutrition 
knowledge?  You  might  also  wish  to  evaluate  efforts  to 
promote  breastfeeding.  Here  questions  might  include: 
What  proportion  of  WIC  mothers  chose  to  breastfeed 
their  infants  before  and  after  the  breastfeeding 
promotion  project  began?  Are  breastfeeding  mothers 
different  (demographically,  for  example)  than  mothers 
who  choose  to  bottle  feed?  Did  involvement  in  the 
breastfeeding  promotion  program  affect  the  length  of 
time  that  women  breastfed  their  infants? 

■    Cost-benefit  analysis  —  the  combining  of  data  from 
both  process  and  impacts  studies  to  compare  the  costs 
of  the  program  with  the  estimated  benefits.  For  our 
nutrition  education  example,  a  cost-benefit  analysis 
would  compare  the  costs  of  providing  nutrition 
education  with  the  observed  benefits  (improved  diet, 
prenatal  weight  gain)  for  WIC  clients.  Results  of  a 
cost-benefit  analysis  often  report  such  findings  as  that 
for  every  x  dollars  spent  on  WIC,  x  dollars  are  saved  in 
health  care  costs. 
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//.  Deciding  Whether  To  Evaluate 


The  first  decision  for  any  evaluation  is  to  determine 
whether  or  not  an  "evaluation"  is  an  appropriate 
method  for  obtaining  needed  information.  Before 
deciding  to  evaluate,  you  should  answer  four 
questions: 

Why  should  we  evaluate? 

What  do  we  want  to  know? 

How  will  we  use  the  results? 

What  resources  are  available? 


What  Do  We  Want  To  Know? 

Next,  you  must  determine  the  scope  of  the  evaluation. 
What  information  do  you  need  to  achieve  the  stated 
objective  of  the  evaluation?  In  more  formal  terms,  the 
task  is  to  develop  research  questions.  When  you 
review  these  questions  you  should  be  able  to  decide 
whether  or  not  your  reason  for  evaluating  merits  the 
effort  that  is  required.  Some  information  may  be 
interesting  but  not  necessarily  useful  to  warrant  the 
expenditure  of  scarce  resources  on  an  evaluation.  On 
the  other  hand,  a  careful  consideration  of  the  research 
questions  may  reinforce  your  original  inchnation  to 
evaluate. 


Why  Should  We  Evaluate? 

If  the  evaluation  is  to  have  a  clear  direction,  you  must 
be  able  to  explain  clearly  why  you  wish  to  study  the 
issue  or  issues  you  have  identified.  If  a  logical 
rationale  cannot  be  stated,  there  may  be  no  need  for 
the  study. 

For  the  most  part,  evaluations  are  conducted  to: 

■  Obtain  information  for  preparing  program  plans 
and  budgets. 

■  Assess  and  improve  the  efficiency  and /or 
effectiveness  of  day-to-day  operations. 

■  Modify  procedures  or  adjust  for  operational 
change,  such  as  fewer  staff  or  larger  caseloads. 

■  Address  specific  issues  such  as  staff  turnover, 
budget  deficits,  or  the  delivery  of  WIC  services. 

Begin  by  setting  out  —  in  writing  —  the  objectives  of 
your  study.  You  may  write  only  a  few  sentences,  but 
the  exercise  of  writing  them  will  help  clarify  your 
purpose.  For  example,  your  study  objective  may  be  to 
determine  why  your  breastfeeding  promotion 
campaign  did  not  increase  the  number  of 
breastfeeding  mothers  in  WIC. 

It  is  important  to  clearly  identify  your  objectives.  Your 
reasons  for  evaluating  will  affect  your  choices  of 
methodology,  data  collection  procedures,  and 
reporting  formats. 


Typical  Research  Questions 

Typical  research  questions  that  may  be 
asked  of  any  program  are: 

How  well  did  the  program  achieve  its 
goals? 

Were  the  program's  activities 
achieved  as  planned? 

For  which  groups  was  the  program 
most/least  successful? 

What  did  the  program  cost? 

How  well  was  the  program  managed? 

Research  questions  can  be  of  several  different  types. 
Descriptive  questions  ask  about  the  characteristics  of 
something,  e.g,  who  is  participating  in  WIC? 
Cause-and-effect  questions  try  to  identify  if,  how,  or 
why  something  occurred,  e.g.,  what  is  the  impact  of 
WIC  on  participants? 

Research  questions  tailored  for  a  specific  study  can  be 
devised  using  a  simple  format  called  an  EPD,  or 
Evaluator's  Program  Description  (exhibit  II-A).  This 
format  developed  by  Fink  and  Kosecoff 
(An  Evaluation  Primer:  Evaluator's  Program 
Description  (c)  1978.  Reproduced  by  permission  of 
Sage  Publications,  Inc.)  offers  a  way  to  develop 
evaluation  questions  that  assess  both  the  impact  of  the 
program  under  study  as  well  as  the  process  or 
operation  of  the  program.  For  example,  assume  that  at 
the  end  of  a  breastfeeding  promotion  campaign,  your 
evaluation  finds  no  clear  increase  in  breastfeeding 


3 


rates.  It  is  not  enough  to  conclude  that  the  promotion 
failed;  you  want  to  know  why  it  failed  so  that  you  do 
not  repeat  mistakes,  and  to  fine-tune  your  approach. 
To  learn  why,  you  must  closely  assess  each  campaign 
activity. 

The  E valuator's  Program  Description  should  include: 

1.  Program  goals  and  activities.  A  program 
goal  is  a  statement  of  intent.  An  activity  is 
a  means  of  achieving  a  goal. 

2.  Evidence  of  program  merit.  What  would  it 
take  to  convince  you  that  the  activity  was 
successfully  achieved?  The  evidence  of 
program  merit  describes  exactly  what 
must  take  place  to  achieve  the  goal  or 
activity. 

3.  Evaluation  questions.  After  you  have 
completed  the  program  description,  you 
are  ready  to  decide  what  questions  the 
evaluation  should  answer.  The  program 
merit  section  is  the  best  source  for 
evaluation  questions  because  it  defines  the 
successful  accomplishment  of  goals  and 
activities. 

You  can  add  to  the  list  of  questions  developed  by  the 
EPD.  Through  specific  questions,  you  can  address 
whether  or  not  a  service  is  being  delivered.  Answering 
the  questions  will  teach  you  a  substantial  amount 
about  the  quality  of  the  service,  the  cost  of  delivery, 
and  the  program's  impact. 

How  Will  We  Use  the  Results? 

Before  you  start  an  evaluation,  it  is  essential  to  identify 
the  intended  uses  of  the  results.  The  planned  use  of 
the  findings  will,  to  some  extent,  determine  the 
primary  research  questions,  set  priorities  on  specific 
research  issues,  and  direct  the  allocation  of  resources. 

Uses  of  evaluation  results  include: 

■  required  reports  to  sponsoring  agencies,  WIC  State 
agencies,  or  FNS 

■  strategic  planning 

■  professional  staff  development 


■  technical  assistance 

■  general  planning  and  budgeting 

■  requests  for  increased  resources 

■  performance  monitoring 

■  local  management  and  administration 

Clearly,  the  use  of  results  should  mesh  with  the  stated 
purpose  for  conducting  the  evaluation. 

What  Resources  Are  Available? 

A  final  question  in  deciding  to  evaluate  is  whether 
there  are  sufficient  resources  to  complete  a  credible 
evaluation.  Consider  the  following: 

■  When  is  the  information  needed?  Is  there 
sufficient  time  to  design  the  study,  collect  the  data, 
analyze  the  results,  and  write  a  report? 

■  Is  there  a  similar  study  that  could  be  replicated  or 
simply  reviewed  to  find  answers  to  the  posed 
research  questions? 

■  Are  data  now  available  (collected  and  ready  for 
analysis)  that  can  be  used  for  the  evaluation? 

■  Are  there  questionnaires  or  other  data  collection 
forms  that  can  be  used  or  adapted  or  will  new 
instruments  have  to  be  designed? 

■  Do  you  have  staff  with  sufficient  time  and  skills  to 
carry  out  all  the  steps  of  this  evaluation? 

■  Is  technical  assistance  available  at  a  cost  the 
agency  can  afford? 

■  Are  there  funds  for  data  collection  or  abstraction, 
computer  time,  analysis,  and  report  publication? 

Any  of  these  considerations  might  dramatically  affect 
your  schedules  or  ability  to  proceed  with  the  planned 
evaluation. 
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EVALUATOR'S  PROGRAM  DESCRIPTION* 


Goal 


One  by  one,  describe  the 
goals  of  the  program 
under  study. 


Activity 


For  each  goal,  describe 
the  program  activity  or 
method  that  will  lead  to 
fulfillment  of  the  goal. 


Evidence  of  program 
merit 

For  each  goal,  describe 
the  type  of  information 
that  will  be  convincing 
evidence  of  program  merit 
or  success. 


Related  evaluation 
questions 

A  question  is  developed  to 
determine  whether  or  not 
each  measure  of  program 
merit  was  met. 


1.  To  increase  the  number 
of  pregnant  women 
enrolled  in  WIC  from  60 
per  month  in  1989  to 
70  per  month  in  1990. 


1.  WIC  coordinator  meets 
with  prenatal  clinic  staff 
to  develop  an 
expedited  referral 
process  for  WIC 
certification. 


1.  WIC  coordinator  met 
with  prenatal  clinic  staff 
to  discuss  an  expedited 
referral  process. 


1.  Did  the  WIC 

coordinator  meet  with 
prenatal  clinic  staff  to 
discuss  a  new  referral 
process?  When?  What 
major  issues  were 
discussed? 


2.  Based  on  discussions 
with  prenatal  clinic 
staff,  WIC  coordinator 
established  a  protocol 
for  the  referral  of 
prenatal  patients  for 
WIC  certification. 


2.  Protocol  was 
established  to  refer 
prenatal  patients  for 
WIC  certification. 


2.  Was  an  expedited 
protocol  established  to 
refer  patients  for  WIC 
certification? 


3.  Certify  prenatal 
applications  within  5 
days  of  initial  contact 
with  the  program. 


3.  All  prenatal  applications 
certified  within  5  days 
of  initial  contact. 


3.  Were  all  prenatal 
applications  certified 
within  5  days  of  initial 
contact?  If  not,  why 
not?  On  average,  how 
many  days  did  it  take  to 
certify  new  applicants? 


These  questions  assess  the  activities  (or  the  process)  of  an  initiative  to  increase  participation  among  pregnant 
women.  This  format  helps  us  assess  the  implementation  of  the  initiative  as  well  as  its  overall  impact.  You  can  add 
to  the  questions  identified  by  the  EPD.  Those  you  may  want  to  add  for  this  example  include: 

1 .  Did  the  number  of  pregnant  women  enrolled  in  WIC  increase  from  60  per  month  in 
1989  to  70  per  month  in  1990? 

2.  Did  participation  increase  for  some  groups  of  pregnant  women  but  not  others? 
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///.  Identifying  Sources  of  Information 


Based  on  your  research  questions,  identify  the  most 
appropriate  data  source(s)  to  ensure  that  the  data 
collected  will  relate  to  the  planned  evaluation.  For 
example,  if  we  choose  to  evaluate  a  special 
breastfeeding  promotion  program  that  is  still 
underway,  the  best  data  source  may  be  the  original 
program  planners  and  designers  rather  than  current 
staff  now  operating  the  breastfeeding  program. 

How  To  Locate  Data 

You  can  obtain  data  from  existing  sources  (extant 
data)  or  from  new  data  sources  (primary  data 
collection).  Sometimes  new  data  is  more  expensive 
to  collect  than  existing  data.  You  should  carefully 
compare  the  "informational"  value  of  new  data 
collection  with  the  costs  of  acquiring  that  information, 
and  decide  if  it  is  worth  the  expense. 

WIC  managers  already  collect  a  wealth  of 
information  for  program  reporting.  Take  the  time  to 
rethink  how  you  use  your  information,  you  may  not 
need  to  collect  additional  information  to  answer 
questions  about  program  effectiveness.  Rearrange 
your  existing  data  and  think  about  it  in  different  ways. 
Assess  changes  in  your  data  over  time,  annually, 
quarterly,  and  look  for  patterns  and  trends. 

Data  available  to  a  local  WIC  administrator  include 
(for  a  list  of  Federal  data  sources,  see  appendix  C): 

1)  client  records  maintained  by  the  local  agency. 
These  include  a  host  of  demographic  data  about 
the  population  you  serve.  Data  from  client  and 
certification  records  can  help  you  assess 
measures  like  the  trimester  in  which  the 
majority  of  pregnant  women  enroll  in  WIC;  the 
percentage  of  low-birthweight  babies  born  to 
WIC  mothers;  the  percentage  of  infants  with 
low  hemoglobin  and  hematocrit  levels  who 
improve  by  recertification,  broken  down  by 
race  and  geographic  location  of  residence. 


2)  automated  food  instrument  issuance  system 
records.  Such  systems  provide  information  on 
clients  and  records  which  enable  you  to  identify 
the  percentage  of  no-shows  for  food  instrument 
pickup  and  the  percentage  of  instruments 
issued  and  not  cashed  broken  down  by  risk 
level.  They  also  provide  you  with  historical 
information  on  food  vendors. 

3)  databases  maintained  by  State  health 
departments  and  vital  records  bureaus.  Public 
data  on  births,  infant  deaths,  and  people  served 
by  a  public  health  system  can  provide  valuable 
information  to  a  WIC  manager. 

4)  State  and  local  population  statistics.  State  and 
city  planning  offices  maintain  census  data  that 
may  be  helpful  in  targeting  special  services  or 
outreach.  In  many  instances,  you  can  obtain 
local  maps  of  your  city  and  identify  the 
lowincome  populations,  infant  populations,  the 
residences  of  various  ethnic  groups,  or  other 
populations  of  interest.  For  a  visual 
representation  of  where  your  services  are 
located,  plot  clinics  on  the  map  as  well  and 
assess  where  you  need  to  target  services. 

5)  financial  records.  Financial  and  expense 
statement  records  from  your  program  or 
contracting  offices  are  good  data  sources. 
Periodic  measurement  of  labor  and  equipment 
costs  can  help  you  assess  the  cost  of  providing 
WIC  services  over  time. 

6)  annual  reports.  Annual  WIC  Program  reports 
can  be  used  to  identify  trends  in  service 
delivery  and  program  costs.  Compare  annual 
reports  from  year  to  year  and  develop  graphs 
to  easily  identify  service  trends. 

You  can  request  that  data  from  other  agency 
information  systems  be  transferred  onto  personal 
computer  disks  for  use  at  a  local  agency.  Before 
planning  new  data  collection,  you  should  talk  with 
State  WIC  staff  as  well  as  other  State  and  local  health 
and  public  welfare  staffs,  at  least  annually,  to 
determine  what  data  are  available. 
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Prior  to  using  and  relying  on  existing  data  for 
program  evaluation,  be  sure  the  data  is  available  in  the 
format  you  need.  Is  it  ready  for  analysis  or  data  entry? 
Does  it  need  to  be  cleaned  or  edited  prior  to  use?  In 
addition,  include  the  staff  that  maintain  the  data  in 
designing  the  study  and  interpreting  findings.  They 
offer  an  insight  that  you  may  not  have  and  can  help 
you  identify  strengths  and  weaknesses  in  the  data. 


How  to  Organize  Your  Data  Needs 

Different  sources  can  provide  answers  to  different 
questions.  You  should  review  each  question  to  identify 
the  information  needed  to  answer  it.  Prepare  a  simple 
three-column  list  (exhibit  III-A)  listing  the  question, 
the  information  needed,  and  the  corresponding 
information  source(s). 


EXHIBIT  III-A 
Three-Column  List  To  Organize  Your  Data  Needs 


Research  Question 


Needed  Information 


Information  Source 


1.  How  many  participants 
breastfed  before  the 
campaign? 


1.  No.  of  breastfeeding 
participants  prior  to 
the  outreach 


1.  Local  agency  files 


2.  How  many  participants 
breastfed  after 
the  campaign? 


2.  No.  of  breastfeeding 
participants  after 
the  outreach 


2.  Local  agency  files 


3.  Did  the  breastfeeding 
promotion  campaign 
make  the  difference? 


3.  Staff  and  participant 
perspectives  on  change 
in  breastfeeding  rates 


3.  Staff  interviews, 
participant  interviews 


Other  efforts  underway 
that  might  affect 
breastfeeding  rates 


Discussions  with 

health  officials 

on  current  initiatives 

that  might  affect 

breastfeeding 

rates 


4.  How  much  did  the 
campaign  cost? 


4.  Estimate  of  labor  and 
other  costs  expended 


4.  Staff  time  estimates, 
salary/benefit  rates, 
cost  of  supplies 
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IV.  Evaluation  Designs 


There  are  many  evaluation  designs  from  which  to 
choose.  The  appropriate  design  depends  on  the  quahty 
and  volume  of  data;  the  evaluation's  focus;  intended 
results;  and  staff,  time,  and  resource  constraints. 
Among  the  typical  evaluation  methodologies  are  case 
studies,  surveys,  field  experiments,  pre/ post  designs, 
observational  studies,  simulation,  and  forecasts. 

■  Case  Studies  —  A  case  study  is  an  analytic 
description  of  single  or  multiple  events,  processes, 
institutions,  or  programs.  It  includes  comprehensive 
descriptions  of  the  key  players  and  their  motives  as 
well  as  of  key  issues.  Case  studies  can  be  exploratory, 
explanatory,  or  descriptive  and  may  lead  to  larger 
evaluations.  It  may  suggest  solutions  to  identified 
problems,  listing  pros  and  cons.  These  studies  may  be 
comprehensive  but  they  often  cannot  be  used  to 
generalize  in  other  situations. 

■  Surveys  —  Cross-sectional  surveys  are  limited  to  a 
descriptive  analysis  of  a  representative  sample  at  one 
point  in  time.  These  surveys  provide  a  "snapshot"  of  a 
large  sample,  so  the  population  under  study  must  be 
well  defined.  Cross-sectional  surveys  require  rigorous 
sampling  procedures  to  ensure  that  the  sample  closely 
resembles  the  population  of  interest.  Panel  surveys  are 
similar  to  cross-sectional  studies  but  collect 
information  acquired  at  two  or  more  points  in  time 
from  the  same  respondents  (e.g.,  infant  weight  gain 
now,  at  6  months,  at  9  months).  Panel  surveys  are 
effective  for  measuring  changes  in  facts,  attitudes,  or 
opinions  over  a  course  of  time.  Sometimes  it  is  difficult 
to  keep  the  "panel"  or  survey  respondents  intact;  this 
design  often  requires  the  evaluator  to  track 
respondents  to  complete  data  collection. 

■  Field  Experiments  —  field  experiments  are  used  to 
prove  cause-and-effect  relationships  (e.g,  nutrition 
education  technique  caused  improved  dietary 
patterns).  Generally,  such  evaluations  are  "controlled" 
in  that  the  researcher  assigns  a  target  population  to  be 
on  the  receiving  end  of  certain  actions  (treatment  or 
experimental  group)  while  a  control  group  will  receive 
none.  The  researcher  compares  the  two  groups  at  the 
end  of  the  experiment.  A  target  population  may  be 
composed  of  persons,  families,  neighborhoods,  towns, 
hospitals,  schools,  and  so  on.  A  measurable  outcome  is 
one  that  can  be  attributed  to  the  specific  policy  and/ or 
action.  Its  presence  or  absence  alone  can  be  noted  for 
each  individual  in  the  study. 


In  a  true  experimental  design,  members  are  randomly 
assigned  to  treatment  and  control  groups;  this  design 
provides  a  persuasive  argument  about  causal  effects 
of  a  program  on  participants.  The  random  assignment 
of  respondents  to  treatment  and  control  groups 
ensures  that  both  groups  are  equal  (across  certain 
characteristics  such  as  age,  race,  area  of  residence)  and 
the  observed  differences  at  the  end  of  the  experiment 
may  be  attributed  to  the  intervention. 

Nonequivalent  comparison  group  designs  are  those  in 
which  respondents  are  not  randomly  assigned  to 
treatment  and  control  groups.  Because  the  groups  may 
be  different  in  some  fashion,  outcomes  cannot  be 
solely  attributed  to  the  intervention.  This  type  of  study 
provides  less  proof  of  cause  and  effect  and  the 
researcher  must  consider  other  factors  that  may  affect 
study  outcomes  beyond  the  intervention  due  to  the 
possibility  of  differences  between  the  two  groups. 

■  Pre/Post  Designs  —  pre/post  designs  compare 
individuals  before  and  after  a  program  or  a  specific 
intervention.  Generally,  no  other  comparison  groups 
are  used  to  help  rule  out  alternative  explanations  for 
outcomes.  Pre/ post  designs  can  measure  change,  but 
the  researcher  cannot  attribute  change  solely  to  the 
program. 

■  Observational  Studies  —  As  the  name  suggests,  the 
researcher  observes  ongoing  activity,  and  gathers  field 
data  as  they  are  available.  The  researcher  does  not 
intervene  with  any  action  or  other  experimental  effort. 
While  this  may  sound  somewhat  informal, 
observational  studies  include  systematic  and  planned 
data  collection  about  the  persons  or  institutions  of 
interest  to  the  researchers.  For  example,  if  a 
breastfeeding  promotion  effort  has  been  operating  for 
2  years  and  is  now  an  accepted  part  of  routine 
operations  at  the  WIC  local  agency,  the  evaluation 
may  seek  to  determine  how  the  promotion  actually 
operates  and  how  it  compares  with  the  originally 
designed  program. 

■  Simulation  —  This  process  uses  a  mathematical 
model,  together  with  input  data,  to  probe  the  response 
of  a  system  to  changes  that  have  not  actually  occurred. 
Sometimes,  the  system  has  not  even  been  created.  For 
example,  proposed  changes  to  the  Aid  to  Families 
with  Dependent  Children  (AFDC)  Program  and  the 
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Food  Stamp  Program  are  often  simulated  on  computer 
models  to  estimate  the  costs  and  effects  of  program 
innovations  or  modifications. 

■    Forecasts  —  A  prediction,  or  forecast,  makes  a 
judgment  about  future  possibilities  by  extrapolating 
historical  experience  into  the  future.  A  projection  (a 
type  of  forecast)  is  a  mathematical  exercise  that 
specifies  a  model  for  what  remains  constant  and  how 
things  change;  specifies  particular  parameter  values 
(or  boundaries);  calculates  the  future  implications  of 
these  trends;  and  reflects  the  most  likely  future  course 
or  range  of  possibilities. 


If  your  key  research  questions  are  descriptive  in 
nature  and  you  need  to  generalize  study 
findings,  cross-sectional  or  panel  surveys  may 
be  appropriate.  If  you  do  not  need  to  generalize 
the  results  of  a  descriptive  study  to  other 
settings,  case  studies  normally  will  do. 

If  your  research  questions  are  cause-and-effect 
questions,  and  there  is  a  need  to  generalize 
findings,  comparison  group  experiments  may 
be  appropriate.  If  there  is  no  need  to  generalize 
findings,  nonequivalent  comparison  groups 
may  be  an  alternative. 


Linking  a  Design  to  the  Evaluation  Questions 

There  are  many  ways  to  design  an  evaluation. 
However,  your  evaluation  questions  are  the  key  to 
what  design  you  should  use.  For  example: 
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V.  Understanding  Sampling  Strategies 


When  the  population  of  interest  is  too  large  to  obtain 
information  on  each  individual  member,  a  sample  is 
drawn.  Sampling  allows  evaluators  to  make 
predictions  about  a  population  based  on  study 
findings  from  a  sample  of  cases.  To  make  accurate 
statements  about  the  population,  a  sample  must  be 
representative  (unbiased)  and  exhaustive  (inclusive  of 
every  possible  category  of  participants)  of  the 
population  under  study.  An  unbiased  sample  is  one 
that  represents  the  population.  For  example,  it  may  be 
appropriate  to  select  every  tenth  case  file  from  1,000 
WIC  records  (for  a  sample  of  100)  for  a  review  of 
average  income  levels  of  WIC  participants.  This  would 
be  an  unbiased  sample  because  cases  are  not  generally 
filed  by  income  status,  family  size,  or  any  other 
pattern  that  would  bias  your  sample  towards  higher 
(or  lower)  income  levels.  This  would  allow  you  to 
generalize  from  your  sample  to  all  1,000  WIC  cases 
(i.e.,  the  population). 

Four  Major  Points  You  Should  Know  About 
Sampling 

Sampling  can  be  a  tricky  task.  If  you  choose  to  sample, 
you  may  need  to  consult  a  professional  statistician  or 
researcher.  Here  are  four  main  points  you  need  to 
know  about  sampling: 

1 .  As  a  rule  of  thumb,  do  not  sample  when  it  is 
just  as  easy  and  not  much  more  expensive  to 
obtain  data  from  everyone  in  the  population. 

2  Do  not  attempt  to  draw  a  sample  to  support 
major  decisions  when  you  are  not  trained  in 
sampling  methods  or  without  consulting  a 
professional. 

3.  There  are  different  types  of  samples. 

Probability  samples  impose  statistical  rules  to 
ensure  that  unbiased  samples  are  drawn.  These 
samples  are  normally  used  for  impact  studies. 
Nonprobability  samples  are  less  complicated  to 
implement  and  less  expensive.  This  type  of 
sampling  is  appropriate  for  most  process  or 
implementation  studies. 


are  less  worried  about  generalization.  As  a  rule 
of  thumb,  the  more  critical  the  decision,  the  larger 
(and  more  precise)  your  sample  should  be. 

Below  are  discussions  on  types  of  samples,  sample 
size,  and  examples. 

Probability  versus  Nonprobability 

Probability  sampling  imposes  mathematical  and 
procedural  rules  that  must  be  followed  to  ensure 
that  an  unbiased,  appropriately  sized  sample  is 
drawn.  The  primary  idea  is  that  every  individual, 
object,  or  institution  (e.g.,  clinic)  in  the  population 
under  study  has  a  chance  of  being  selected  into  the 
sample  and  the  likelihood  of  the  selection  of  any 
individual  is  known.  This  is  attained  through  some 
process  of  randomization.  Probability  sampling 
provides  a  firmer  basis  for  generalizing  from  the 
sample  to  the  population  or  universe.  Because  the 
requirements  for  drawing  this  type  of  sample  are  quite 
precise,  technical  assistance  —  either  from  published 
materials  or  from  persons  with  substantive  and 
methodological  expertise  —  may  be  needed. 

It  is  possible  and  sometimes  appropriate  to  use  a 
nonprobability  sample.  This  is  less  complicated  to 
conduct  and  less  expensive  than  drawing  a  probability 
sample.  It  is  wise  to  draw  as  large  a  sample  as 
possible.  Reliance  on  nonprobability  samples  reduces 
the  researcher's  ability  to  generalize  the  results  of  the 
study  to  the  larger  population. 


4.  There  are  tradeoffs  you  must  make  when 
determining  sample  size.  Large  samples  are 
more  precise  than  smaller  ones,  yet  they  can  be 
expensive  to  carryout.  Small  samples  are  okay  if 
you  are  looking  for  a  big  change  or  effect  and 
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TYPE  OF  SAMPLES 

Probability  Sampling  (the  following 
probability  sampling  designs  have  the 
characteristic  of  random  selection  in  one  way 
or  another): 

■  Random  . . .  Each  element  from  the 
target  population  has  an  equal  chance  of 
selection.  Usually,  tables  of  random  numbers 
are  used  to  draw  a  random  sample.  The 
researcher  may  pick  a  starting  point  anywhere 
in  the  table  and  move  in  any  direction.  Once 

a  starting  point  is  selected,  the  number  of 
columns  of  digits  must  be  sufficient  to  give 
each  element  in  the  population  a  chance  of 
selection.  For  example,  if  there  are  900 
elements  in  the  population,  three  columns  of 
random  numbers  are  needed  because  there 
are  three  digits  in  "900. "  To  make  the  random 
selections,  the  elements  of  the  universe  are 
numbered,  and  the  selection  of  random 
numbers  from  the  table  continues  until  the 
desired  sample  size  is  reached. 

■  Stratified  random  ...  a  refinement  of  the 
simple  random  sample,  this  divides  a 
population  into  similar  groups  or  strata  (based 
on  identifiable  and  important  characteristics). 
All  strata  are  included  in  the  sample,  and 
elements  are  selected  randomly  from  each 
strata.  For  example,  if  we  are  conducting  a 
study  of  children  on  WIC,  to  ensure  that  we 
include  a  proportional  number  of  male  and 
female  children  in  our  study,  first  we  divide  all 
children  into  two  groups  based  on  sex  and 
then  randomly  select  from  each  group.  The 
number  of  males  and  females  selected 
depends  on  the  number  of  each  sex  in  the 
population. 

■  Cluster . . .  another  type  of  probability 
sampling,  this  involves  dividing  a  population 


into  subgroups  (clusters)  instead  of 
individuals  and  then  randomly  selecting  some 
of  the  clusters  for  the  study.  Once  a  cluster 
is  randomly  selected  for  the  sample,  all 
individuals  in  that  cluster  are  included  in  the 
sample.  Cluster  sampling  is  most  effective 
when  the  clusters  are  internally  diverse  —  the 
opposite  of  stratified  random  sampling. 
Cluster  sampling  is  typically  used  in  survey 
research  to  minimize  data  collection  costs. 
An  example  would  be  to  randomly  select 
neighborhoods  of  a  city  and  then  survey  all 
WIC  clients  who  live  in  the  selected 
neighborhoods. 

■  Systematic ...  a  technique  often  used 
to  study  records,  this  requires  creating  a 
strategy  for  choosing  the  sample.  The 
random  selection  of  the  first  sample  member 
determines  the  entire  sample.  For  example,  a 
systematic  sample  could  include  every  record 
that  ends  in  the  number  seven.  This  sampling 
approach  is  effective  and  simple  so  long  as 
the  method  for  assigning  numbers  to  the 
records  under  study  does  not  depend  on  any 
characteristic  of  the  record  itself. 

Nonprobability  Sampling: 

MConvenience . . .  where  information  is 
obtained  from  available  subjects.  For 
example,  a  sample  of  all  breastfeeding 
women  who  are  certified  or  who  pick  up 
food  instruments  during  the  month  of  April. 

■  Quota  ...  a  form  of  convenience 
sampling  where  information  is  obtained  on 
some  proportion  (or  quota)  of  the  population 
of  interest  to  the  researcher.  For  example,  50 
percent  of  all  pregnant  clients  who  receive 
nutrition  education  services. 
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Most  process  or  implementation  studies  do  not  require 
strict  probability  sampling.  Impact  studies,  on  the 
other  hand,  which  quantitatively  measure  program 
effects  with  implications  for  the  population,  generally 
require  a  probability  sample. 

Sample  Size 

The  size  of  the  sample  depends  on  several  factors; 
most  importantly,  how  small  an  effect  you  want  to 
detect  and  how  important  it  is  that  you  do  not  miss 
the  effect  if  it  occurs.  There  are  tradeoffs  between  large 
and  small  samples.  Small  samples  (in  relation  to  the 
size  of  the  population)  are  economical  and  convenient; 
large  samples  are  more  reliable  and  representative  of 
the  population  under  study.  The  researcher  must 
balance  practical  considerations  against  statistical 
power  and  generalizability  when  determining  sample 
size.  Other  factors  which  effect  sample  size  include: 

■  Larger  samples  will  give  you  greater  certainty  and 
more  statistical  precision.  If  you  are  basing  a  major 
decision  on  data  you  are  collecting,  then  you  will  want 
precision  and  certainty.  Statistical  precision  increases 
as  the  sample  size  increases. 

■  When  there  is  considerable  variability  in  the 
population  you  need  a  larger  sample.  When  the 
population  is  homogeneous,  a  smaller  sample  is 
appropriate  to  adequately  represent  the  population. 

■  A  larger  sample  will  cost  more  time  and  money. 
The  larger  the  sample,  the  more  resources  it  takes  to 
collect  the  data.  You  must  ask  yourself,  is  the  increase 
in  certainty  worth  the  cost? 

■  If  you  expect  that  a  number  of  individuals  will  not 
respond  or  participate  in  the  study,  you  may  need  to 
over  sample. 

■  When  differences  in  the  results  are  expected  to  be 
small,  a  larger  sample  guards  against  "noise"  or  other 
confounding  variables  that  might  contribute  to  the 
results  of  a  treatment. 

■  For  exploratory  research  and  pilot  studies, 
samples  between  10  and  30  are  used  as  a  rule.  These 
samples  are  quick  and  convenient  to  work  with  and 
they  are  large  enough  to  test  a  hypothesis,  yet  small 
enough  to  overlook  weak  treatment  effects. 


Sample  Size  and  Generalization 

Sampling  allows  you  to  make  predictions  or  reach 
conclusions  about  a  population  based  on  study 
findings  from  a  sample  of  cases.  Making  statements 
about  a  larger  group  (or  a  population)  is  referred  to  as 
generalization.  However,  the  extent  to  which  you  can 
"generalize"  or  infer  that  study  findings  from  a  sample 
are  true  for  an  entire  population  depends  on  the  size 
and  make-up  of  the  sample.  In  order  to  generalize, 
a  sample  must  be  representative  (unbiased)  and 
exhaustive  (inclusive  of  every  possible  category  of 
participants)  of  the  population  under  study. 

Responsible  research  does  not  draw  conclusions 
about  "large"  groups  based  on  "small"  studies.  As 
you  consider  sampling  strategies,  think  about  how 
you  will  use  the  study  results.  Don't  "stretch" your 
findings;  use  sound  sampling  strategies  and  logic 
to  guide  you  when  interpreting  and  reporting  study 
results. 


Example:  Nonprobability  Sampling  Size: 

A  local  WIC  administrator  may  wish  to  collect 
some  quantitative  information  on  one  or 
several  specific  topics  but  may  not  have  the 
resources  to  draw  a  large  probability  sample. 
Using  nutrition  education  as  an  example, 
let's  assume  that  the  average  annual  WIC 
caseload  is  2,000  (1,000  women,  750 
infants,  and  250  children).  Using  information 
from  the  State's  automated  information 
system,  you  learn  that  the  agency  certifies 
an  average  of  167  clients  each  month,  with 
more  certifications  occurring  in  the 
second  quarter  of  the  calendar  year.  You 
decide  to  conduct  the  evaluation  during  the 
second  quarter  when  you  have  the 
opportunity  to  acquire  information  on  more 
than  one-third  of  the  caseload  receiving  the 
service  (nutrition  education).  While  this 
sample  is  not  a  probability  sample,  it  is  a 
substantial  proportion  (or  quota)  of  the 
targeted  population. 
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There  are  different  ways  to  collect  data,  and  different 
corresponding  collection  instruments.  Some  of  the 
methods  and  instruments  are  listed  below: 


other  studies.  It  may  be  feasible  to  use  an  existing 
instrument  or,  at  much  less  cost,  to  adapt  a 
field-tested  and  validated  one. 


Collecting  Method 

1.  Abstracting — data  obtained  from  existing 
source  such  as  a  client  records 

2.  Interviewing — information  obtained  in 
person  from  individuals.  Interviews  are 
good  for  collecting  qualitative  or  sensitive 
data  such  as  values  or  attitudes. 


3.  Surveying — information  acquired  by  mail, 
telephone,  or  in-person  interview,  by 
asking  individuals  precisely  the  same 
questions  in  a  standardized  format. 

4.  Observing — data  accumulated  by  observers 
who  watch  certain  activities. 


5.  Recording — data  collected  on  occurrence, 
frequency,  and/or  specific  indicators  about 
one  or  more  activities. 


Collection  Instrument 

Abstraction  forms.  Data  collectors  complete 
a  form  for  each  record  reviewed. 

Interview  guides  or  questionnaires.  These 
can  be  structured,  where  respondents  are 
asked  the  same  questions  in  a  specific 
order,  or  semistructured,  where  topics  are 
listed  and  can  be  asked  in  the  order  that 
works  best  for  each  respondent 

Survey  questionnaires.  Always  structured  so 
that  each  respondent  is  asked  the  same 
questions  usually  in  a  specific  order;  often 
self  administered. 

Observation  forms.  Generally  created  as 
checklists  or  grids  so  that  observers  can 
quickly  record  what  they  see. 

Other  recording  forms.  Can  be  matrices, 
grids,  checklists,  and  so  on.  Their  design  is 
dependent  on  the  type  of  information  that  is 
collected. 


Designing  a  data  collection  instrument  can  be  a 
complex  undertaking  but  it  need  not  be  if  you  take  the 
time  to  think,  plan  carefully,  and  follow  these  suggestions: 

■  The  items  that  comprise  an  instrument  must  be 
mutually  exclusive;  different  from  each  other;  and 
collectively  exhaustive.  The  instrument  must  be 
designed  to  acquire  all  the  data  needed  for  the 
specific  evaluation. 

■  The  instrument  should  be  as  brief  as  possible. 
Clearly,  the  primary  objective  is  to  collect  the 
necessary  data,  so  it  is  important  to  minimize  the 
burden  placed  on  respondents. 

■  Because  instrument  design  may  be  difficult, 
time-consuming,  and  expensive,  check  on 
instruments  designed  and  tested  in  the  field  for 


Going  back  to  the  hypothetical  examination  of  the 
breastfeeding  promotion  outlined  in  section  III, 
several  data  collection  instruments  are  needed.  Simply 
modify  the  existing  list  of  needed  information  and 
information  sources,  expanding  it  to  four  columns  in 
order  to  continue  designing  the  evaluation  (exhibit 
VI- A). 

Before  designing  the  instruments,  check  the 
Compendium  of  WIC  Research  (appendix  A)  to  find 
out  whether  any  other  State  or  local  agency  has 
conducted  a  similar  evaluation  of  its  breastfeeding 
promotion  program.  If  so,  call  the  contact  person  to 
find  out  what  instruments  were  used  and  how;  and 
arrange  to  receive  copies  of  these  forms.  Even  if  an 
evaluation  did  not  deal  exclusively  with  breastfeeding 
promotion,  its  instruments  may  provide  a  good  format 
and  direction. 
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EXHIBIT  Vl-A 


Four-Column  List  to  Organize  Data  Needs 
and  Accompanying  Instruments 


Research  Question 

1.  How  many  participants 
breastfed  before  the 
cannpaign? 

2.  How  many  participants 
breastfeeding 

after  the  campaign? 

3.  Did  the  breastfeeding 
promotion  campaign 
make  a  difference? 


Needed  Information 

1.  No.  of  breastfeeding 
participants  prior  to 
the  outreach 

2.  No.  of  breastfeeding 
participants  after 
the  outreach 

3.  Staff  and  participant 
perspectives  on  change 
in  breastfeeding  rate 


Information  Source 


1.  Local  agency  files 


2.  Local  agency  files 


Staff  interviews, 

participant 

interviews 


Type  of  Instrument 

1 .  Record  abstraction 
form 


2.  Record  abstraction 
form 


3.  Staff  survey, 
participant 
survey 


4.  How  much  did  the 
campaign  cost? 


4.  Estimate  of  labor  and 
other  costs  expended 


4.  staff  time  estimates, 
salary/benefit  rates, 
cost  of  supplies 


4.  Staff  time  logs, 
staff  survey, 
cost  survey 


Training  Data  Collectors 

It  is  likely  that  you  will  need  to  train  people  to  obtain 
your  information.  Obviously,  the  particulars  of  training 
will  depend  on  the  collection  method,  the  data  to  be 
obtained,  and  the  skills  and  backgrounds  of  the 
collection  team.  Because  resources  are  usually  limited, 
the  State  or  local  WIC  administrator  may  have  limited 
options  with  regard  to  data  collectors.  However,  you 
should  try  to  find  people  who  are  interested  and 
unbiased  —  that  is,  who  do  not  have  a  preconceived 
notion  of  study  results.  Each  individual  must  be  trained 
to  collect  precisely  the  information  you  have  identified 
and  defined.  The  data  must  be  comparable  across  all 
data  collectors;  this  is  particularly  necessary  for 
quantitative  studies  of  program  impacts.  However,  it  is 
also  important  in  process  studies. 

As  with  data  collection  instruments,  other  agencies  may 
have  prepared  accompanying  training  manuals,  too.  Be 
sure  to  ask  about  training  manuals  when  calling  your 


colleagues  about  existing  instruments.  In  addition,  a 
local  WIC  administrator  could  contact  the  State  WIC 
agency,  other  State  agencies,  local  colleges,  or  schools  of 
public  health  for  assistance.  Graduate-level  students  are 
an  excellent  resource  because  they  often  need  to  collect 
data  for  master-level  courses  and  more  advanced 
degrees. 

Pilot  Testing 

Whenever  possible,  instruments  should  be  pilot-tested. 
That  is,  they  should  be  administered  to  a  small  number 
of  respondents  before  full-scale  data  collection  begins. 
This  will  help  identify  any  problems,  inconsistencies, 
or  inaccuracies  so  that  instruments  can  be  revised. 
The  pilot  test  will  also  reveal  whether  needed  data 
can  be  obtained,  how  long  it  takes  to  administer  a 
questionnaire,  and  the  appropriateness  of  the  wording 
of  individual  items  or  questions.  It  will  also  provide 
training  for  the  data  collectors. 


14 


VII.  Analyzing  The  Data 


Analysis  and  assessment  are  inherently  judgmental 
processes  that  should  be  guided  by  the  original 
evaluation  objectives  and  professional  standards 
commonly  used  by  the  evaluation  and  research 
community.  An  information  source  that  might  be 
useful  to  WIC  administrators  in  choosing  analytic 
methods  is  "The  Evaluation  Framework,"  a 
publication  of  the  U.S.  General  Accounting  Office. 

Qualitative  Data 

Process  evaluations  (discussed  earlier  in  section  1) 
often  involve  the  analysis  of  qualitative,  descriptive 
data.  Qualitative  data  includes  descriptive  information 
on  such  things  as  staff  opinions  on  program  redesign 
or  the  quality  of  services  from  a  participant's 
perspective.  The  researcher  collects  data  and  looks  for 
common  themes,  patterns,  or  hypotheses.  In  essence, 
qualitative  analysis  may  not  include  counting  but 
rather  summarizing  and  interpreting  study  findings. 
Qualitative  data  can  strengthen  and  further  explain 
quantitative  findings.  Analyzing  qualitative  data 
usually  requires  substantial  knowledge  of  the  program 
as  well  as  a  thorough  understanding  of  the  tasks 
comprising  day-to-day  delivery  of  program  services. 
An  evaluator  who  assesses  a  large  volume  of 
qualitative  data  may  wish  to  incorporate  an  advisory 
panel  into  the  analysis  portion  of  the  evaluation  who 
can  provide  a  broad  view  and  consensus. 

Quantitative  Data 

More  formally  structured  analysis  methods  are 
applied  to  quantitative  or  numerical  data.  These 
methods  range  from  simple,  easy-to-calculate 
descriptive  statistics  (discussed  below)  to 
sophisticated  regression  analyses  such  as  probit 
and  logit. 

Measures  of  Central  Tendency 

There  are  various  statistical  techniques  to  analyze 
particular  types  of  data.  An  evaluator  must  first 
determine  which  type  of  data  is  available  for  analysis. 
Usually,  an  evaluator  who  analyzes  quantitative  data 
first  calculates  descriptive  statistics.  Primary  among 
such  statistics  are  measures  of  central  tendency  which 
represent  typical  or  "average"  values  of  the  variables 
under  study.  A  measure  of  central  tendency  is 
valuable  because  it  captures,  in  a  single  number, 
information  on  an  entire  population  or  sample. 
Measures  of  central  tendency  are  defined  below. 


■  The  mode  is  the  value  or  item  occurring  most 
frequently  in  a  series  of  observations  or  statistical 
data.  The  mode  can  be  used  with  any  type  of 
data.  If,  for  instance,  WIC  records  indicate  the 
distribution  of  WIC  breastfed  infants  is  200  black 
infants,  400  white,  and  100  Hispanic,  then  the 
modal,  or  "typical,"  breastfed  infant  is  white. 

■  The  median  is  the  value  that  divides  a 
distribution  into  two  parts  of  equal  area,  or  the 
50th  percentile,  that  is,  half  the  measurements  are 
on  one  side  of  the  median  and  half  are  on  the 
other  side  when  all  measurements  are  arranged  in 
order  of  magnitude.  The  median  is  unrelated  to 
any  arithmetic  manipulation  of  values  and  is 
simply  the  middle  one.  For  example,  if  there  are 

5  children  whose  ages  are:  2,  2,  2,  2,  5,  the  median 
age  is  2  years;  this  value  is  the  middle  one  of  all 
the  values  selected.  The  value  of  the  statistic 
depends  on  the  order  or  rank  of  the  data  and  tells 
us  more  about  the  entire  group  than  the  mode 
does. 

■  The  mean,  or  arithmetic  average,  is  the  sum  of 
the  values  divided  by  the  total  number  of  values. 
It  is  the  most  commonly  used  summary  measure. 
Using  the  data  on  ages  for  the  example  of  a 
median,  sum  all  of  the  ages,  then  divide  by  five 
(the  number  of  cases  in  the  sample).  The  mean 
age  of  this  group  of  children  is  2.6  years.  The 
difference  between  the  mean  and  mode  (2  versus 
2.6)  is  that  the  mean  is  more  sensitive  to  extreme 
numbers  (the  5  year  old)  and  reflects  the  value  of 
every  member  in  the  group. 

Since  each  measure  of  central  tendency  provides 
different  information  you  can  specify  all  three 
averages  for  a  comprehensive  description  of  a  group. 

Measures  of  Dispersion  (Variability) 

It  is  also  useful  to  calculate  measures  of  dispersion, 
or  "variability,"  which  indicate  the  level  of  spread  of 
a  group.  Two  such  measures  which  may  be  of  use  to 
WIC  administrators  are: 

■  Range.  The  range  for  numerical  data  is  the 
difference  between  the  largest  and  smallest 
observed  values  +  1  (range  =  highest  -  lowest  +  1). 
The  range  for  categorical  data  is  the  number  of 
categories  in  the  continuum,  for  example  from 
"very  satisfied"  to  "very  dissatisfied." 
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■  Variance.  The  variance  describes  how  close  to 
a  central  measure,  or  how  far  away,  the  values  of  a 
distribution  are.  The  standard  deviation  is  often 
used  to  identify  how  elements  of  a  group  differ 
from  the  group  mean.  The  measure  of  central 
tendency  is  more  useful  if  the  variance  is  small; 
that  is,  the  values  are  not  spread  out. 


Frequencies  and  Cross-Tabulations 

These  are  other  measures  that  can  be  easily  calculated. 
A  frequency  is  a  simple  count  of  the  number  of  times 
something  occurs.  A  cross-tabulation  is  a  table  of 
counts  arranged  into  distributions.  Note  the 
accompanying  example  of  a  cross-tab  of  a  WIC 
agency's  caseload  of  pregnant  women  by  age  and 
race/  ethnicity. 


WIC  Caseload  of  Women  Participants 
by  Age  and  Race/ Ethnicity 

Race 

Asian  or  American  Indian 
Age                              White              Black           Hispanic           Pacific  or  Alaskan 

Islander  Native 


14to  19 

22 

19 

12 

2 

9 

20  to  24 

46 

38 

23 

12 

14 

25  to  29 

34 

30 

19 

14 

11 

30  to  34 

20 

15 

12 

10 

8 

35  and  older 

7 

5 

5 

4 

6 

DATA  ANALYSIS: 
Breastfeeding  Promotion  Example 

To  complete  the  evaluation  for  the 
breastfeeding  promotion  example 
used  earlier  (in  the  3-  and  4-column 
lists)  requires  an  analysis  of 
quantitative  data  on  changes  in 
breastfeeding  rates.  Cross-tabulations 
of  the  breastfeeding  participants 
before  and  after  the  campaign  may  be 
reported  in  a  frequency  table.  It  may 
be  interesting  to  break  this  information 
out  by  age  and  race.  Identify  the 
mean,  mode,  and  median  and  a 
measure  of  variability  of  the  age  of  a 


breastfeeding  client  before  and  after 
the  campaign  to  assess  whether  you 
were  successful  in  outreaching  to  a 
different  population. 

Identify  the  costs  of  implementing  the 
campaign  in  terms  of  labor  hours  and 
costs  of  equipment  and  materials. 
Based  on  your  results,  was  it  worth  it? 

The  qualitative  information  compiled 
from  the  interviews  may  be  used 
to  prepare  a  summary  on  the 
participant  and  staff  perspectives 
on  the  effectiveness  and  opinions 
of  the  breastfeeding  campaign. 
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VIII.  Reporting  Evaluation  Results 


How  you  plan  to  use  evaluation  results  will  clearly 
shape  reporting  options.  If  an  evaluation  is  conducted 
at  the  request  of  a  State  or  Federal  agency,  it  is  likely 
that  certain  requirements  have  been  established  for 
reporting  results.  If,  however,  it  was  a  local  decision  to 
evaluate,  then  you  should  develop  a  reporting  format 
early  in  the  process  when  you  consider  the  intended 
use  of  results.  Factors  to  consider  in  planning  and 
preparing  a  report  include: 

■  Audience.  Who  will  read  or  hear  this  report? 
Why  are  they  interested  in  the  results? 

■  Type  of  Report.  Is  a  single,  formal  report 
appropriate?  Should  a  summary  version  be 
prepared  and  distributed?  Are  there  other 
outlets  (professional  newsletters,  for  example) 
that  should  be  used  for  disseminating  results? 

■  Resources.  Be  sure  to  set  aside  sufficient 
resources  for  the  kind  of  reporting  you  intend. 
For  example,  resources  may  be  needed  to  hire 
a  report  writer  or  editor,  type  mailing  lists, 
duplication,  postage,  etc. 

What  To  Include  in  Your  Evaluation  Report 

The  report  format  you  choose  depends  on  the 
audience.  At  a  minimum,  an  evaluation  report  should 
include: 

■  an  introduction  that  describes  your  study 
objectives  and  your  key  research  questions. 

■  a  description  of  your  study  design  and 
sampling  strategies.  Be  sure  to  include  what 
type  of  study  it  is  (process,  impact),  and  your 
methodology  (case,  comparative  evaluation, 
etc.).  If  you  sampled  participants  or  objects  for 
the  study,  explain  what  sampling  technique 
was  used  (random,  quota,  etc.,)  and  sample 
size  (intended  and  eventually  realized).  Also 
be  sure  to  describe  the  total  population  under 
study. 

■  a  description  of  your  data  collection 
procedures  (who  collected  the  data,  when, 
and  where)  and  a  full  description  of  your 
instruments  as  well  as  copies. 

■  a  description  of  analyses  techniques. 


■  evaluation  findings.  Answer  each  research 
question  and  include  any  recommendations 
and  interpretations  of  findings.  Also,  report 
both  negative  and  positive  findings.  Negative 
findings  may  help  identify  areas  where 
improvements  can  be  made.  They  should  not 
be  ignored  or  viewed  as  failures  of  your 
program  or  staff.  And  always  consider 
alternative  explanations  for  study  results!! 

■  a  summary  of  problems  encountered  during  the 
evaluation  and  a  full  acknowledgment  of  any 
limitations  of  the  data. 


Using  Illustrative  Charts  and  Graphs 

Charts  and  graphs  are  effective  ways  to  summarize 
your  findings.  Several  illustrations  that  may  be  useful 
are: 

■  Bar  charts  illustrate  comparisons  between 
items.  They  may  be  either  vertical  or  horizontal. 
Sorting  items  from  largest  to  smallest  can  be 
very  effective.  Segmented  or  stacked  bar  charts 
illustrate  how  the  whole  is  made  up  of  its 
component  parts.  This  format  depicts  how  the 
components  change  over  time.  A  clustered  bar 
chart  can  show  how  a  set  of  items  has  changed 
over  time  (see  examples). 


Segmented  or  Stacked  Bar  Chart 

Percent 

120  r- 

100  -  I  1 

80  - 
60  - 


Women 


I    I  Pregnant  ^  Breastfeeding  |  Postpartum 

Total  Women  Participation  in  WIC 
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Cluster  Bar  Chart 


Pie/Bar  Chart 


Category 


Women 


Infants 


Children 


0       20  40 
WIG  Caseload  1988-1989 


60       80      100      120     140  160 

Caseload 


Infants 


Children 


Women 


WIC  Caseload 


Pregnant 

Postpartum 
Breastfeeding 


Pie-bar  charts  are  a  combination  of  the  above 
and  can  be  effective  when  one  portion  of  a 
whole  has  several  subcomponents. 


■  Pie  charts  illustrate  how  the  whole  is  divided 
up  into  parts.  They  allow  you  to  emphasize  the 
percentage  of  each  segment  of  a  population. 
You  can  explode  a  slice  of  a  pie  to  emphasize  a 
certain  segment  of  a  population. 


Several  graphic  software  packages  are  available  on  the 
market.  Some  familiar  ones  are  Lotus  Graphwriter, 
Harvard  Graphics,  and  Desk  Top  Publisher.  For  those 
that  have  access  to  such  tools,  evaluation  reports  with 
a  professional  appearance  can  be  produced  on  a 
limited  budget.  Most  of  the  packages  are  also  easy  to  use. 


Sample  Pie  Chart 

"Cut  or  Exploded"  Slice 


White 
20.0% 


Racial/Ethnic  Breakdown  of  Caseload 
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Appendix  A 

Studies  of  thie  WIC  Program  by  Independent  Evaluators 
and  State  and  Local  WIC  Agencies 


Compendium  of  WIC  Research 

This  section  presents  brief  descriptions  of  several 
studies  of  the  WIC  Program.  Some  of  the  evaluations 
and  other  research  projects  were  conducted  by  State 
and  local  WIC  agencies  while  others  are  the  work  of 
academic  or  independent  researchers.  The  matrix 
below  lists  each  study  described  in  this  compendium. 


The  evaluation  summaries  may  prove  useful  to  WIC 
administrators  in  conducting  their  own  evaluations. 
The  descriptions  include  sources  for  obtaining  copies 
of  evaluations  or  materials  to  use  for  an  evaluation. 
Each  research  effort  includes  the  title  of  the  research, 
researchers,  citation  (if  applicable),  research  focus, 
population  studied,  research  objectives,  study  design, 
analytic  methods  used,  study  findings,  intended 


Study  Name 


Process  Impact  Cost-  Benefit 

Analysis  Analysis  Analysis 


Alabama  Breastfeeding  Study 

Massachusetts  Study 
on  Pregnancy  Outcomes 

Missouri  Prenatal  Study 

North  Carolina  Study  on 

WIC  Effects  on  Health  Services 


X 
X 


Missouri  WIC/Medicaid  Study 


X 


New  Jersey  Study  of 
Breast-  feeding  Promotion 

New  Jersey  Study  of  Nutrition 
Education  Promotion 


X 


Tennessee  Evaluation 

of  Breast-  feeding  Promotion 

California  Study  of  Anemic  Children 

Washington,  DC,  Study 
of  Targeted  Outreach 


X 
X 

X 


Oregon  Market  Research  Study 
Minnesota  WIC  Outreach  Plan  1987 
Wisconsin  Standard  Deviations  Report 


X 

X* 
X* 


*  While  these  studies  are  not  formal  or  traditional  process  analyses,  they  are  descriptive  presentations  that 
may  be  useful  to  other  WIC  practitioners. 
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audience,  and  applicability  for  State  and  local  WIC 
administrators. 

The  studies  are  examples  of  State  and  local  evaluations 
voluntarily  submitted  by  attendees  at  the  1990 
meeting  of  the  National  Association  of  WIC  Directors. 

Studies  of  the  WIC  Program 
by  Independent  Evaluators 

The  first  five  evaluations  are  State-specific.  However, 
this  group  of  studies  was  conducted  by  independent, 
academically  based  evaluators  or  by  evaluators  from 
State  agencies  other  than  State  (or  local)  WIC  agencies. 
Four  of  the  evaluations  are  impact  analyses;  one  is  a 
cost-benefit  study. 

Impact  Analyses 

Perceptions  and  Knowledge  of  Breast-Feeding 
Among  WIC  and  Non-WIC  Pregnant  Women  in 
Alabama 

Terence  R.  Collins,  James  D.  Leeper,  Tamar  Milo,  and 
Sarah  DeMellier 

Citation:  The  Alabama  Journal  of  Medical  Sciences. 
1984  21  (2):  145-148 

Research  Focus:  Impact  of  WIC  participation,  age,  and 
race  on  pregnant  women's  decisions,  perceptions,  and 
knowledge  about  breastfeeding 

Population  Studied:  Prenatal  WIC  participants  in  six 
Alabama  counties 

Research  Objectives:  The  primary  purpose  of  the 
study  was  to  explore  the  intention  to  breastfeed  and 
the  factors  influencing  that  decision  in  the  WIC 
population  as  compared  with  these  same  factors  in 
a  demographically  similar  group  of  non-WIC  women. 

Study  Design:  Trained  interviewers  conducted 
structured  interviews  with  341  WIC  participants 
and  a  comparison  group  of  178  non-WIC  clients  of 
public  health  departments.  Data  were  collected  on 
demographics,  pregnancy  history,  infant-feeding 
preferences,  health  care,  and  health  service  utilization. 
The  comparison  group  was  demographically  similar 
to  the  WIC  group. 

Analytic  Methods:  Descriptive  statistics  were  computed 


on  all  demographic  data.  Evaluators  also  calculated 
proportions  of  women  planning  to  breastfeed  as  well 
as  proportions  of  responses  for  the  reasons  given 
for  infant-feeding  choices.  Chi-square  tests  were 
used  to  identify  statistically  significant  proportional 
differences  between  the  treatment  and  comparison 
groups.  Statistically  significant  differences  between  the 
means  of  continuous  variables  were  determined  using 
two  independent  sample  T-tests.  Yates  correction  was 
used  with  the  chi-square  tests  when  both  dimensions 
had  two  categories. 

Study  Findings:  There  was  no  significant  difference 
between  plans  to  breastfeed  by  WIC  and  by  non-WIC 
women  (18.5  percent  versus  16.3  percent).  In  general, 
women  chose  the  method  they  perceived  as  more 
convenient.  Among  women  not  planning  to 
breastfeed,  a  larger  percentage  of  WIC  participants 
said  formula  was  "just  as  good"  (66.2  percent  versus 
51.3  percent)  and  also  said  that  they  did  not  know  how 
to  breastfeed  (30.2  percent  versus  20  percent).  Women 
aged  30  and  older  were  more  likely  to  breastfeed  than 
younger  women;  white  women  were  more  likely  to 
breastfeed  than  were  black  women.  The  authors 
conclude  that  the  Alabama  WIC  Program  "...  could 
do  a  better  job  of  influencing  women  to  choose 
breastfeeding." 

Intended  Audience:  Public  health  practitioners, 
educators,  and  evaluators,  including  the  WIC 
community 

Applicability:  As  noted  in  the  article,  this  study  was 
limited  to  a  consideration  of  only  prenatal  intentions. 
Postpartum  behavior  was  not  studied.  With  sufficient 
resources,  a  State  WIC  agency  could  undertake  a 
similar  study. 

WIC  Participation  and  Pregnancy  Outcomes: 
Massachusetts  Statewide  Evaluation  Project 

Milton  Kotelchuck,  Janet  B.  Schwartz,  Marlene  T. 
Anderka,  and  Karl  S.  Finison 

Citation:  American  Journal  of  Public  Health.  1984  74 
(10):  1086-1092 

Research  Focus:  Impact  of  WIC  participation  on 
birthweight,  gestational  age,  morbidity,  and  infant 
mortality 
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Population  Studied:  WIC  prenatal  participants  and  their 
infants 

Research  Objectives:  This  study  addressed  four 
questions: 

■  Does  the  WIC  Program  reach  its  target 
population? 

■  Is  WIC  participation  associated  with  more 
positive  outcomes  of  pregnancy? 

■  Are  the  effects  of  WIC  participation  similar 
across  all  high-risk  subpopulations? 

■  Are  the  effects  of  the  WIC  Program  enhanced 
with  increased  duration  of  participation? 

Study  Design:  As  a  comparison  group,  this  study  used 
a  matched  pairs  design  in  which  the  birth  outcomes  of 
4,126  pregnant  women  who  participated  in  the  WIC 
Program  and  who  gave  birth  in  1978  were  compared 
with  the  birth  outcomes  of  4,126  women  who  did  not 
participate  in  WIC.  The  study  controlled  for  maternal 
age,  race,  parity,  education,  and  marital  status.  Data 
were  obtained  from  the  State's  WIC  management 
information  system  as  well  as  from  the  State's  vital 
statistics  records. 

Analytic  Methods:  For  1978,  demographic 
characteristics  of  WIC  participants  were  contrasted 
with  the  characteristics  of  all  pregnant  women 
residing  in  the  same  catchment  area  and  across  the 
State.  WIC  participants  were  compared  directly  with 
their  matched  non-WIC  controls  on  birth  outcome 
measures.  Statistical  differences  were  examined 
with  paired  T-test  comparisons  and  chi-square 
comparisons.  Pairwise  deletions  were  used  for  any 
subject  pair  with  missing  data.  Using  demographic 
characteristics  and  duration  of  WIC  participation, 
both  WIC  and  control  women  were  stratified  into 
subpopulations  for  separate  analyses  of  birth  outcome 
differences. 

Study  Findings:  WIC  prenatal  participants  are  at 
greater  demographic  risk  for  poor  pregnancy 
outcomes  compared  with  all  women  in  the  same 
community.  WIC  participation,  however,  is  associated 
with  improved  pregnancy  outcomes:  a  decrease  in 
low-birthweight  incidence  (6.9  percent  versus  8.7 
percent)  and  neonatal  mortality  (12  versus  35  deaths). 


an  increase  in  gestational  age  (40.0  versus  39.7  weeks), 
and  a  reduction  in  inadequate  prenatal  care  (3.8 
percent  versus  7.0  percent).  Analysis  of  the  stratified 
subgroups  indicates  that  subpopulations  at  higher  risk 
(teenagers,  unmarried  women,  and  Hispanic-origin 
women)  have  "more  enhanced"  pregnancy  outcomes 
associated  with  WIC  participation.  Review  of  the 
stratifications  based  on  duration  of  WIC  participation 
shows  that  increased  participation  is  associated  with 
enhanced  pregnancy  outcomes.  Despite  the  positive 
results,  the  authors  caution  that  there  are  other  factors 
which  might  distinguish  between  the  two  groups 
and  which  could  serve  as  the  basis  for  alternative 
explanations.  (See  Applicability  below.) 

Intended  Audience:  Public  health  practitioners, 
educators,  and  evaluators,  including  the  WIC 
community 

Applicability:  This  study-one  of  the  earliest  studies  of 
WIC  impact-demonstrates  the  positive  findings  with 
regard  to  birth  outcomes  that  several  researchers  have 
attributed  to  WIC  participation.  However,  it  also  notes 
some  difficulties  of  conducting  WIC  research  such 
as  the  problem  of  selection  bias.  That  is,  in  studies  of 
WIC,  researchers  must  confront  the  problem  of 
self-selection—some  women  choose  to  participate  in 
WIC  and  they  may  be  more  strongly  motivated  to 
improve  the  prenatal  health  of  their  babies  than  are 
women  in  the  control  population.  Such  a  motivational 
difference  could  cause  both  an  improvement  in 
pregnancy  outcome  and  a  desire  to  enroll  in  the  WIC 
Program. 

Evaluation  of  the  Missouri  WIC  Program: 
Prenatal  Components 

Joseph  W.  Stockbauer 

Citation:  Journal  of  the  American  Dietetic  Association. 
1986  86:  61-67 

Research  Focus:  Impact  of  WIC  participation  on 
birthweight  and  gestational  age 

Population  Studied:  Missouri  WIC  prenatal  participants 
delivering  babies  during  1980  (8.5  percent  of  Missouri 
mothers  delivering  in  1980) 

Research  Objectives:  This  study  addresses  three 
questions: 
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■  Does  prenatal  nutrition  supplementation 
influence  the  outcome  of  pregnancy? 

■  Does  length  of  participation  in  prenatal 
nutrition  supplementation  influence  the 
outcome  of  pregnancy? 

■  Are  certain  risk  categories  used  as  criteria 
for  program  participation  more  likely  to 
be  influenced  by  prenatal  nutrition 
supplementation  than  others? 

Study  Design:  The  study  employed  two  data  sources: 
the  Missouri  WIC  client  information  system  which 
provided  data  on  prenatal  participants  between 
October  1979  and  June  1981  and  on  all  infants  enrolled 
in  WIC  who  were  born  during  1980;  and  Missouri 
birth  and  fetal  death  certificates.  The  data  merge 
resulted  in  6,732  matches  which  formed  the  study 
population.  The  actual  study  population  was  slightly 
smaller  because  fetal  deaths  and /or  twin  births  were 
excluded.  Data  on  birthweight  and  gestational  age  for 
WIC  and  non-WIC  groups  were  analyzed. 

Analytic  Methods:  Three  methods  of  analysis  were 
employed  to  tackle  the  central  problem  of 
self-selection  (selection  bias)  and  the  resulting  lack 
of  randomization: 

1.  Acquisition  of  a  random  sample  of  6,560  births 
from  live-born  singletons  delivered  to  non-WIC 
Missouri  residents  during  1980.  Analysis  of 
covariance  was  applied  to  statistically  equate 
the  two  groups. 

2.  Use  of  the  1980  Missouri  resident  live  births 
minus  the  WIC  study  group  births  as  a 
comparison  group  and  use  of  the  WIC  group  as 
the  standard  population,  adjusting  for  key 
variables  (such  as  gravidity,  complications  of 
pregnancy)  related  to  the  outcome  measures. 

3.  Development  of  a  pair-match  study  group  from 
the  remainder  of  the  1980  Missouri  resident 
data  set  matching  on  key  variables  (race,  age, 
education,  smoking  during  pregnancy, 
prepregnancy  weight,  and  number  born  this 
pregnancy). 

Stiidx/  Findings:  Overall,  this  study  showed  that  WIC 
prenatal  nutritional  supplementation  has  a  positive. 


though  not  conclusive,  impact  on  reducing  low 
birthweight  and  raising  mean  birthweight.  A  higher 
mean  birthweight  was  noted  for  WIC  births  and  for 
the  WIC  nonwhite  group.  In  either  instance,  the 
amount  of  increase  was  small.  A  reduced  low  birth 
weight  rate  was  noted  for  all  WIC  participants  as  well 
as  for  the  WIC  nonwhite  group  regardless  of  the 
method  of  analysis;  the  differences  were  statistically 
significant  for  the  standardization  method.  Duration 
in  WIC  had  a  positive  influence  on  both  mean 
birthweight  and  low  birthweight,  regardless  of  race. 

Nonwhite  WIC  participants  recorded  a  statistically 
significant  lower  percentage  of  births  occurring  prior 
to  37  weeks  gestation  than  the  corresponding 
non-WIC  group.  Non-white  WIC  participants  also 
recorded  a  lower  perinatal  death  rate  than  the 
corresponding  non-WIC  group  after  adjustments 
were  made  for  education  and  length  of  pregnancy. 

A  lower  low-birthweight  rate  was  recorded  for  the 
majority  of  WIC  risk  categories.  However,  only  the 
WIC  medical  risk  group  had  a  statistically  significant 
lower  rate. 

Intended  Audience:  Public  health  researchers 
particularly  nutritionists  and  practitioners  especially 
in  the  WIC  community 

Applicability:  This  study  employed  sophisticated 
statistical  methodologies  that  require  specific  skills 
and  training  as  well  as  the  resources  to  complete  the 
computer  merges  on  fairly  large  data  sets.  A  State  WIC 
agency  may  possess  such  resources  or  might  be  able  to 
obtain  help  from  staff  in  vital  statistics  sections.  The 
article  provides  sufficient  detail  to  allow  for 
replication. 

Effect  of  a  WIC  Program  on  Children's  Clinic 
Activity  in  a  Local  Health  Department 

Jonathan  B.  Kotch  and  David  Whiteman 

Research  Focus:  Impact  of  WIC  participation  on  an 
increase  in  the  demand  for  services  in  a  local  health 
department 

Population  Studied:  Children  in  Columbus  County,  NC, 
who  used  the  services  of  a  children's  health  clinic 
during  1978  and  1979 
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Study  Design:  A  computerized  system  was  used  to 
track  encounter  data  in  a  children's  health  clinic.  The 
unit  of  measurement  for  this  study  is  the  encounter 
not  the  child,  and  5,983  encounters  occurred.  An 
encounter  is  a  visit  to  the  health  clinic.  Child  visits  for 
health  screening,  health  maintenance,  acute  illness, 
and  foUowTip  care  w^ere  included  in  the  total  number 
of  encounters.  Encounters  were  classified  into  three 
types:  WIC;  well  baby/well  child  ("well");  and  all 
other  encounters  ("other").  The  WIC  category  does  not 
include  nutrition  education  or  counseling.  A  separate 
analysis  tracked  individual  children  who  first 
appeared  at  a  WIC  clinic  but  who  later  visited  a 
non-WIC  clinic.  Only  children  born  on  or  after  July  1, 
1977,  were  included  in  the  this  analysis.  This  group  of 
1,258  infants  accounted  for  2,946  (or  about  half)  of  the 
encounters. 

Analytic  Methods:  Evaluators  calculated  the  frequency 
of  all  clinic  encounters  including  WIC  visits.  These 
data  were  also  analyzed  quarterly  and  were  further 
broken  into  the  three  encounter  categories.  The 
frequency  of  encounters  processed  by  the  clinic's 
nursing  staff  was  also  computed.  Finally,  encounter 
frequencies  for  infants  in  the  subanalysis  were  also 
calculated. 

Study  Findings:  WIC  clinic  visits  were  found  to 
disproportionately  contribute  to  children's  health 
clinic  activity.  Nearly  80  percent  of  the  increase  in  total 
clinic  encounters  was  attributable  to  WIC.  In  the  first 
quarter  of  fiscal  year  1978,  only  18  percent  of  infant 
encounters  in  the  children's  clinic  were  for  WIC 
services;  that  number  rose  to  more  than  58  percent  by 
the  last  quarter  of  fiscal  1979.  During  the  same  period, 
the  number  of  children  on  WIC  as  a  percentage  of 
all  non-WIC  encounters  with  public  health  nurses 
increased  from  19  to  34  percent.  Infants  who  came  for 
the  first  time  to  the  children's  clinic  to  be  certified  for 
WIC  contributed  an  increasing  percentage  of  all 
infants  seen  in  non-WIC  clinics — from  0  percent  at  the 
start  of  the  study  to  almost  15  percent  by  the  final 
quarter. 

The  authors  conclude  that  the  presence  of  a  WIC 
program  in  this  local  health  department  led  to 
substantial  increases  in  non-WIC  health  clinic  activity 
that  are  not  financially  supported  by  the  WIC 
Program.  To  further  substantiate  this  point,  Columbus 
County  data  were  compared  with  1978-79  data  from 
two  other  North  Carolina  counties  without  WIC 


programs  but  matched  with  Columbus  on  geographic 
location,  total  population,  infant  mortality  rate, 
percent  of  population  under  5  years  of  age,  percent 
nonwhite  population,  and  percent  of  population 
receiving  AFDC  benefits.  Increases  in  the  total  nursing 
units  of  service  of  infants  and  children  were  7  and  16 
percent  for  the  two  counties  as  compared  with  34 
percent  for  Columbus  County.  Eliminating  WIC 
encounters,  nursing  activity  in  the  Columbus  County 
health  clinic  continued  to  show  a  large  increase-25 
percent. 

Intended  Audience:  Public  health  administrators  and 
practitioners,  legislators,  WIC,  and  other  public  health 
service  policymakers 

Applicability:  The  authors  state  that  previous 
studies  as  well  as  the  Congress  and  WIC  Program 
and  policy  decisionmakers  have  failed  "to  anticipate 
the  additional  demands  that  WIC  would  make  on 
already  overcrowded  health  department  services, 
particularly  on  those  provided  by  public  health 
nurses."  This  study  attempted  to  rectify  this  omission. 
The  authors  suggest  that  "consideration  must  be  given 
to  providing  the  additional  resources"  needed  to 
handle  increased  clinic  visits  for  WIC  services  and  for 
non-WIC  services  used  by  WIC  infants. 

A  State  or  local  WIC  agency  attempting  to  carry  out  a 
similar  study  would  need  the  software  to  operate  the 
computerized  program  which  allowed  for  analysis  of 
encounter  data.  The  analysis  itself  is  relatively 
straightforward  with  perhaps  the  most  complicated 
task  being  the  calculation  of  unduplicated  counts  of 
participating  infants.  However,  in  areas  where  WIC 
programs  are  part  of  local  health  departments, 
regional  or  State  health  departments  might  consider 
conducting  a  similar  study  to  ascertain  the  impact  of 
WIC  on  the  provision  of  public  health  services. 

Cost-Benefit  Analyses 

WIC  Prenatal  Participation  and  Its  Relationship  to 
Newborn  Medicaid  Costs  in  Missouri:  A  Cost/ Benefit 
Analysis 

Wayne  F.  Schramm 

Citation:  American  Journal  of  Public  Health.  1985  75 
(8):  851-857 
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Research  Focus:  Cost/benefit  analysis  of  WIC 
participation  costs  and  the  associated  reduction  in 
newborns'  Medicaid  costs 

Population  Studied:  Missouri's  Medicaid-covered  WIC 
prenatal  participants  and  their  infants—only  Medicaid 
claims  with  a  date  of  service  within  30  days  of  birth 
and  only  babies  born  in  calendar  year  1980  were 
included  in  the  study  population. 

Research  Objectives:  This  study  addressed  five 
questions: 

■  Does  WIC  participation  reduce  Medicaid  costs 
for  the  newborn  and  its  mother  for  the  30  days 
immediately  following  birth? 

■  Does  WIC  increase  birthweight  and  reduce  low 
birthweight  among  Medicaid  births? 

■  How  do  the  relationships  of  birthweight,  length 
of  hospital  stay,  and  admissions  affect  the 
WIC/non-WIC  Medicaid  cost  differentials? 

■  Is  increased  participation  in  WIC  associated 
with  reduced  Medicaid  costs  and /or  increased 
birthweight? 

■  Do  reduced  Medicaid  costs  for  WIC 
participants  outweigh  WIC  costs,  thus 
demonstrating  a  cost-beneficial  program? 

Study  Design:  This  study  required  linking  four 
separate  data  files:  Medicaid  (medicaid  cost  data); 
birth  certificates  (maternal  characteristics, 
birthweight);  WIC  records  (receipt  and  amount  of  WIC 
benefits),  and  Neonatal  Intensive  Care  Unit  (NICU) 
admissions  (data  for  explaining  differences  between 
WIC  and  non-WIC  Medicaid  costs).  The  evaluation 
used  7,628  Missouri  Medicaid  records  matched  with 
their  corresponding  1980  birth  records.  This  file  was 
then  merged  with  WIC  data  and  divided  into  a  WIC 
group  containing  1,883  records  and  a  non-WIC 
comparison  group  of  5,745  records. 

Analytic  Methods:  For  infants  born  in  1980, 
demographic  and  behavioral  characteristics  of  WIC 
participants  were  contrasted  with  similar  data  for  the 
control  group.  After  adjusting  for  higher  medical  costs 
in  metropolitan  areas,  T-tests  were  used  to  establish 


statistical  differences.  The  total  study  population  was 
used  as  the  standard  population.  As  a  validity  check 
on  the  primary  method  of  adjustment,  an  analysis  of 
covariance  was  performed  using  the  per  diem  hospital 
reimbursement  as  the  covariate. 

Study  Findings:  WIC  participation  was  found  to  be 
associated  with  a  reduction  of  almost  $100  per 
participant  in  Medicaid  costs  for  newborns.  Mothers' 
Medicaid  costs  were  not  affected.  For  every  dollar 
spent  on  WIC,  about  $0.83  in  Medicaid  costs  were 
saved  for  the  period  within  30  days  of  birth. 
Reductions  in  low-birthweight  rates  and  NICU 
admission  rates  among  WIC  infants  are  two  possible 
explanations  for  the  reduced  Medicaid  costs  associated 
with  WIC  food  supplementation.  As  WIC  food  costs 
increased,  both  mean  birthweight  and  newborn 
Medicaid  savings  also  increased.  The  author  notes 
that  only  short-term  savings  were  examined  and  that 
an  examination  of  longer  term  costs  might  produce 
estimates  of  greater  savings. 

Intended  Audience:  Public  health  researchers.  Medicaid, 
and  WIC  policy  and  program  decisionmakers 

Applicability:  The  Missouri  Medicaid  study  was  one  of 
the  first  WIC  evaluations  to  control  for  income,  as  all 
Medicaid  clients  have  low  incomes.  The  author  points 
out  that  this  study  has  the  obvious  methodological 
problems  of  any  retrospective  study  including 
selection  bias.  Further,  nutritional  risks  cannot  be 
easily  identified  on  birth  records,  where  recorded 
risks  are  more  likely  to  be  obstetrical. 

The  study  used  large  data  sets,  complicated  computer 
data  merges  and  matches,  and  fairly  sophisticated 
statistical  methods.  Most  local  WIC  agencies  lack  the 
resources  to  carry  out  such  a  study.  A  State  WIC 
agency  may  possess  such  resources  or  might  be  able  to 
obtain  help  from  staff  in  vital  statistics  sections. 

Evaluations  by  State 
and  Local  WIC  Agencies 

Eight  studies  conducted  by  State  and  local  WIC 
agencies  are  described  in  this  section.  Five  of  these 
were  conducted  by  State  WIC  agencies.  Five  are 
impact  evaluations;  one  is  a  market  research  effort; 
another  is  an  outreach  (benefit-targeting)  plan;  and 
one  is  a  quarterly  statistical  compendium.  These  less 
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formal  evaluations  are  included  because  they  provide 
useful  information  for  State  and  local  WIC  staff  and 
because  they  could  be  easily  replicated  by  other  State 
WIC  agencies. 

Impacts  Analyses 

Effectiveness  of  Inducement  Factor  in  Promoting 
Breast  Feeding  in  a  WIC  Clinic 

Prema  Achari 

Citation:  Plainfield  WIC  Program,  Plainfield,  NJ 

Research  Focus:  Impact  of  the  use  of  an  incentive  to 
promote  breastfeeding  among  WIC  mothers 

Population  Studied:  WIC  prenatal  and  postnatal 
participants  in  Plainfield,  NJ 

Research  Objectives:  This  study  determined  whether  or 
not  offering  incentives  (decorated  mugs)  to  prenatal 
WIC  participants  would  increase  the  likelihood  of 
breastfeeding. 

Study  Design:  The  incentive  offer  was  introduced  in 
December  1985  during  prenatal  classes  on  nutrition 
education  which  also  offered  information  on  the 
nutritional  value  and  importance  of  breast  milk. 
Women  who  decided  to  breastfeed  their  babies  for  at 
least  2  months  were  given  decorative  mugs.  The 
distribution  of  the  mugs  began  in  February  1986.  The 
test  period  was  January  through  December  1986.  The 
comparison  period  was  January  through  December 
1985.  At  recertification,  a  questionnaire  was 
administered  to  mothers  to  obtain  information  on 
breastfeeding. 

Analytic  Methods:  Time-series  data  from  the 
questionnaires  were  analyzed  to  calculate  the  percent 
of  WIC  participants  who  were  breastfeeding  their 
infants  during  the  comparison  period  and  during 
the  test  period.  Month-by-month  differences  in 
breastfeeding  rates  were  calculated  for  the  two 
periods. 

Study  Findings:  During  the  test  period,  the  number  of 
mothers  breastfeeding  their  infants  was  16  to  18 
percent  higher  than  during  the  comparison  period.  In 
all  but  1  month,  the  test  period  rates  exceeded  the 
comparison  period  rates.  Breastfeeding  rates  during 
the  test  period  were  generally  between  17  and  19 


percent.  Rates  during  the  comparison  period  were 
generally  between  14  and  17  percent. 

Intended  Audience:  Administrators  and  staff  in  other 
WIC  agencies,  other  public  health  professionals 
delivering  prenatal  services. 

Applicability:  This  type  of  promotional  campaign  and 
a  similar  study  could  be  replicated  at  other  local 
WIC  agencies.  In  carrying  out  such  an  effort,  WIC 
administrators  and  their  staffs  should  carefully 
separate  test  and  comparison  groups,  define  the  test 
period  as  only  the  time  during  which  an  incentive  is 
offered,  and  more  closely  track  members  of  the 
treatment  group. 

A  Novel  Technique  to  Increase  Attendance 
in  WIC  Nutrition  Education  Classes: 
A  Promotional  Approach 

Prema  Achari 

Citation:  Plainfield  WIC  Program,  Plainfield,  NJ 

Research  Focus:  Impact  of  a  promotional  program 
to  increase  nutritional  awareness  among  WIC 
participants 

Population  Studied:  WIC  participants  in  Plainfield,  NJ 

Research  Objectives:  The  study  examined  the  impact 
of  a  promotional  program  on  five  aspects  of  WIC 
participant  behavior: 

■  Nutritional  awareness  and  understanding 
of  the  value  of  various  foods 

■  Participant  interest  in  nutrition  and  attention 
span  during  nutrition  education  classes 

■  Pickup  rates  for  food  instruments  and 
attendance  at  secondary  nutrition  education 
classes 

■  Purchase  of  WIC  approved  food  items 

■  Nutrition  consciousness  after  termination 
from  WIC 

Study  Design:  The  intervention  (the  technique  to 
increase  attendance)  was  giving  magnetic  models 
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of  WIC  food  items  for  participants  to  use  on  their 
refrigerators.  Participants  received  one  magnet  every  2 
months  (for  a  total  of  eight  magnets)  when  they  came 
to  the  WIC  chnic  to  pick  up  their  food  instruments 
and  to  attend  nutrition  education  classes.  A  pretest 
questionnaire  (on  food  instrument  pickup,  nutritional 
knov^ledge  and  awareness,  and  buying  compliance) 
was  administered  to  300  participants  prior  to  the 
intervention.  One  year  after  the  promotional  campaign 
began,  a  post-test  questionnaire  (covering  the  same 
topics  as  the  pretest  instrument)  was  administered  to 
306  participants. 

Analytic  Methods:  Using  data  from  the  pretest 
and  post-test  questionnaires,  descriptive  statistics 
(percentages  of  participants  missing  food  instrument 
pickup  and  attending  nutrition  classes)  were 
computed  and  compared.  Responses  to  questions 
about  buying  compliance  were  also  reviewed. 

Study  Findings:  Responses  on  the  post-test 
questionnaire  showed  that  participants  liked  the 
magnets  and  believed  the  magnets  helped  them 
to  remember  to  buy  WIC  foods.  Participants  also 
reported  using  the  magnets  to  explain  healthy  eating 
habits  to  their  children.  Attendance  at  the  secondary 
nutrition  education  classes  increased  from  17  percent 
before  the  study  began  to  31  percent  during  the 
post-study  period.  No  data  concerning  the  other 
research  objectives  are  presented  in  the  published 
report. 

Intended  Audience:  Administrators  and  staff  in  other 
WIC  agencies,  other  public  health  professionals 
delivering  educational  services 

Applicability:  This  type  of  intervention  and  a  similar 
study  could  be  replicated  at  other  local  WIC  agencies. 
Copies  of  the  questionnaires  can  be  obtained  from  the 
Plainfield  WIC  Program. 


Evaluation  of  Tennessee  Breastfeeding  Promotion 
Project  (preliminary,  unpublished  results) 

Minda  Lazarov 

Citation:  Tennessee  Breastfeeding  Promotion  Project, 
Tennessee  Department  of  Health  and  Environment, 
Nashville,  TN  37219-5402 


Research  Focus:  Analyze  the  impact  of  a  breastfeeding 
promotion  project  on  breastfeeding  rates  among 
post-partum  women  in  Tennessee 

Population  Studied:  Tennessee  women  who  entered 
Maternal  and  Child  Health  (MCH)  prenatal  projects 
and/ or  the  WIC  Program  during  their  pregnancies 
Research  Objectives:  To  determine  whether  a 
breastfeeding  promotion  project  in  Tennessee 
increased  the  incidence  and  duration  of  breastfeeding 
among  low-income  (MCH  and  WIC)  women  in  five 
rural  counties  and  one  urban  county. 

Study  Design:  In  1987,  the  Tennessee  Department 
of  Health  and  Environment  (TDHE)  initiated  an 
intensive  breastfeeding  promotion  project  in  five 
Tennessee  counties.  Promotional  efforts,  which  were 
conducted  by  TDHE  State  and  regional  staff,  were 
based  on  a  TDHE  manual.  Breastfeeding  Promotion: 
A  Handbook  for  Health  Professionals.  Women 
entering  the  MCH  prenatal  projects  and /or  the  WIC 
Program  during  pregnancy  were  asked  to  participate 
in  a  series  of  educational  activities  designed  to 
increase  the  incidence  of  breastfeeding.  The  activities 
included  a  survey  of  attitudes  about  breastfeeding, 
classes  on  initiating  breastfeeding,  peer  group 
discussions,  individual  counseling  sessions,  and  an 
incentive  effort.  Activities  implemented  to  increase 
duration  of  breastfeeding  included  hospital  and  home 
visits,  telephone  foUowup,  a  new  mother  gift 
incentive  program  (T-shirts,  diapers,  nursing  pads, 
nursing  brassieres),  a  pump  loan  program,  and 
mother-to-mother  support  groups.  An  additional 
survey  of  mothers  in  the  support  groups  was  also 
conducted. 

Data  on  breastfeeding  rates  were  obtained  at  birth  and 
when  infants  were  2  weeks,  6  weeks,  and  3  months 
old.  At  present,  data  from  only  one  county  have  been 
analyzed.  The  information  now  available  does  not 
include  copies  of  questionnaires,  descriptions  of 
survey  conduct,  or  information  on  data  collection. 

Analytic  Methods:  Descriptive  analysis  (frequencies 
and  demographic  proportions)  of  data  from  before 
and  after  the  breastfeeding  promotional  program. 
Available  information  does  not  describe  the 
comparison  population. 

Study  Findings:  The  preliminary  analysis  considered 
data  from  Fayette  County,  a  poor,  rural  county  with 
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a  large  black  population.  In  Fayette  county,  the 
breastfeeding  rates  among  an  unidentified  population 
prior  to  the  promotion  were  17  percent  in  the  hospital 
and  6  percent  at  6  weeks  post-partum.  Breastfeeding 
rates  for  1989  were  41  percent  in  the  hospital  and  22 
percent  at  6  weeks  post-partum.  It  is  unclear  from  the 
report  whether  these  rates  are  only  for  MCH  and 
WIC  participants  or  whether  these  rates  encompass  a 
larger  population.  In  1989,  of  the  women  who  were 
breastfeeding,  two-thirds  of  these  women  were 
breastfeeding  exclusively  while  in  the  hospital,  and 
half  were  still  breastfeeding  exclusively  at  6  weeks 
post-partum.  TDHE  is  planning  to  analyze  data  from 
the  other  counties  in  the  promotional  program. 

Intended  Audience:  Public  health  (particularly  MCH 
and  WIC)  practitioners  and  educators  who  are 
concerned  with  increasing  the  incidence  and  duration 
of  breastfeeding  among  low-income  mothers. 

Applicability:  The  breastfeeding  promotion  as  well  as 
the  evaluation  could  be  conducted  by  other  State  WIC 
agencies  or  local  agencies  with  sufficient  resources. 
More  detailed  information  would  be  needed  in  order 
to  conduct  a  similar  study.  Such  information  can  be 
obtained  from  staff  of  the  Tennessee  Breastfeeding 
Promotion. 


An  Evaluation  of  the  Effect  of  Iron-specific 
Education  on  Mild  Anemia  Among  Children  in  a 
WIC  Program 

Stephen  Plank,  Kristine  Cassidy,  Marta  McKenzie 

Citation:  Department  of  Public  Health,  Shasta  County, 
CA 

Research  Focus:  Evaluation  of  the  impact  of  an 
iron-specific  supplemental  education  component  at 
a  local  WIC  program  on  the  iron  content  of  anemic 
children. 

Population  Studied:  Mildly  anemic  children  in  a  local 
WIC  program 

Research  Objectives:  The  study  assessed  the  impact  of 
an  educational  component  of  WIC  designed  for 
parents  of  anemic  children  in  addition  to  the  standard 
WIC  nutrition  education  program.  The  measures  used 
to  monitor  the  educational  intervention  impact  were 
dietary  iron  intake  and  hematocrit  values. 


Study  Design:  Local  health  care  providers  referred 
to  WIC  all  eligible  children  aged  1  to  5  months  with 
mild  anemia.  The  providers  collected  dietary 
information  and  blood  samples  at  the  time  of  referral. 
The  participating  physicians  agreed  not  to  give 
iron-specific  nutrition  education  or  iron  compound 
prescriptions  before  obtaining  followup  blood  samples 
at  6  months  after  referral  to  WIC. 

The  57  youngsters  referred  were  randomly  assigned 
to  the  treatment  and  control  groups.  Treatment  group 
members  received  the  standard  WIC  nutrition 
education  program  plus  a  1 -minute  audiovisual 
presentation  on  the  importance  of  iron  in  nutrition, 
a  pamphlet  on  the  same  material,  and  individual 
counseling  on  iron-rich  foods.  The  control  group 
members  received  everything  the  treatment  group  did 
except  the  iron-related  nutritional  information.  Both 
groups  received  WIC  food  instruments  for  seven 
iron-rich  foods. 

After  6  months,  children  in  both  groups  were 
reassessed  by  the  health  care  providers.  Dietary 
histories  were  repeated  and  hematocrits  were  again 
taken. 

Analytic  Methods:  Changes  in  mean  daily  dietary  iron 
intake  and  mean  hematocrit  values  for  each  group 
were  compared  using  a  two-tailed  statistical  test 
to  measure  the  impact  of  the  special  iron-related 
nutrition  education  and  counseling  on  these  outcome 
measures. 

Study  Findings:  Both  groups  had  increased  their 
dietary  iron  intake,  but  only  the  increases  reported 
for  the  treatment  group  were  statistically  significant. 
On  the  other  hand,  while  both  groups  had  higher 
hematocrit  values,  the  hematocrit  values  for  the 
control  groups  showed  a  larger  increase  than  for  the 
treatment  group.  These  increases  were  not  statistically 
significant  for  either  group. 

The  researchers  used  the  contradictory  evidence  from 
the  two  measures  (dietary  iron  intake  and  hematocrit 
values)  to  conclude  that  the  treatment  group  parents 
learned  from  the  special  education  effort  (that  is,  they 
learned  which  foods  to  include  in  their  children's 
dietary  histories)  but  that,  given  the  lack  of  change  in 
hematocrit  values,  they  failed  to  apply  these  lessons 
in  everyday  life.  In  fact,  the  evidence  could  be 
interpreted  to  suggest  that  the  special  education  effort 
had  no  more  effect  than  the  standard  WIC  nutrition 
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education  along  with  the  tailored  food  instruments 
which  the  control  group  also  received. 

Intended  Audience:  State  and  local  WIC  policymakers 
and  program  administrators 

Applicability:  The  authors  argue  that  the  study  findings 
suggest  that  the  special  iron-related  nutrition 
education  does  nothing  by  itself  to  improve  the  health 
of  anemic  children  and  question  the  State's  policy  of 
encouraging  health-care  providers  to  offer  nutrition 
education  rather  than  elemental  iron  supplements. 

The  experimental  design  is  sound.  However,  the 
findings  do  not  include  an  assessment  of  the  quality  of 
the  intervention. 


Evaluation  of  Targeted  Outreach  Effort  in  the  Special 
Supplemental  Food  Program  for  Women,  Infants, 
and  Children 

Geraldine  Tompkins 

Citation:  Department  of  Human  Services,  Government 
of  the  District  of  Columbia 

Research  Focus:  Evaluation  of  local  WIC  Program 
efforts  to  target  outreach  to  higher  risk  groups  who 
are  eligible  for  WIC  benefits 

Population  Studied:  WIC  priority  groups 

Research  Objectives:  The  objective  of  the  study  was 
to  assess  the  effectiveness  of  a  targeted  outreach 
campaign  designed  to  increase  the  proportion  of 
high-risk  groups  enrolled  in  WIC.  The  targeted 
outreach  campaign  included: 

■  Direct  mailings  to  parents  of  newborn  babies 
and  to  pregnant  women  insured  by  Medicaid. 

■  Public  service  announcements  and  news 
releases  in  newspapers  and  on  radio  stations. 

■  Distributing  flyers  and  posters  about  WIC  in 
public  housing  and  in  recreation  department 
offices. 

■  Displaying  WIC  posters  on  buses  in 
neighborhoods  with  higher  concentrations  of 
potentially  eligible  women  and  infants. 


Study  Design:  The  study  was  designed  as  a  pre/post 
evaluation.  WIC  promotional  materials  (both  before 
and  after  the  targeted  outreach  campaign)  contained 
a  hotline  number  to  call  for  more  information.  The 
hotline  was  in  the  WIC  district  office  where  calls 
were  answered  by  clerical  staff.  These  staff 
administered  telephone  questionnaires  to  collect  data 
on  demographic  characteristics  of  callers,  referral 
sources,  health  care  arrangements,  and  WIC  status. 
The  questionnaire  was  administered  during  the 
4-month  period  before  targeted  outreach  began  and 
for  over  a  year  after  the  outreach  campaign  was 
initiated.  Demographic  referral  source  data  collected 
precampaign  were  compared  with  data  collected 
during  and  after  the  benefit  targeting  campaign. 

Data  on  WIC  status  were  also  examined  to  compare 
WIC  enrollment  during  the  period  immediately  before 
targeted  outreach  began  with  WIC  enrollment  during 
and  after  the  outreach  campaign. 

Analytic  Methods:  Using  data  from  the  questionnaire 
and  from  WIC  records,  the  study  used  simple 
statistical  frequencies  to  describe  pre /post  outreach 
campaign  differences. 

Study  Findings:  The  study  reported  pre/ post  outreach 
campaign  differences  in  types  of  calls,  referral  sources, 
and  WIC  enrollments  including: 

■  During  the  4-month  period  before  targeted 
outreach,  53  hotline  callers  were  pregnant 
women,  36  were  mothers  wanting  to  enroll 
their  infants,  5  were  breastfeeding  mothers, 
and  5  wanted  to  enroll  their  children  in  WIC. 
Within  1  month  after  the  campaign  began,  56 
of  the  callers  wanted  to  enroll  their  infants  in 
WIC,  22  were  pregnant  women,  18  wanted  to 
enroll  their  children,  and  1  was  a  breastfeeding 
mother. 

■  Before  targeted  outreach,  about  twice  as  many 
callers  had  been  referred  by  a  friend  as  by  a 
physician.  All  other  sources  accounted  for 
about  25  percent  of  the  calls.  During  the 
implementation  of  targeted  outreach,  friends 
remained  the  primary  referral  source — about  25 
percent  of  the  calls.  Physicians  became  a  more 
important  referral  source  accounting  for  about 
23  percent  of  all  calls. 
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■  The  average  number  of  pregnant  women  who 
were  new  enrollees  during  the  first  5  months  of 
the  outreach  campaign  was  25  percent  greater 
than  the  number  of  new  pregnant  enrollees 
during  the  previous  6  months. 

■  The  average  number  of  infants  enrolled 
increased  by  7  percent  over  the  same  period. 

■  The  proportion  of  WIC  enrollees  in  the  three 
highest  priority  groups  increased  from  63.5 
percent  of  total  enrollees  during  the  fiscal  year 
preceding  outreach  to  71 .6  percent  during  the 
fiscal  year  when  outreach  was  initiated  and 
increased  to  78.8  percent  during  the  first  full 
year  after  implementation. 

Researchers  concluded  that  the  outreach  campaign 
was  effective  in  increasing  the  proportion  of  high-risk 
enrollees. 

Intended  Audience:  WIC  Program  administrators 

Applicability:  The  successful  implementation  of  a 
targeted  outreach  campaign  is  of  interest  to  State  and 
local  WIC  administrators.  The  methods  used  in  the 
District  of  Columbia  could  also  be  used  in  other  towns 
and  cities.  The  relative  suddenness  of  the  change  in 
types  of  hotline  callers  suggests  that  the  targeted 
outreach  was  effective.  However,  it  is  important  to 
note  that,  while  WIC  enrollments  exhibited  change 
after  outreach,  this  study  did  not  identify  any  other 
external  factors  or  administrative  or  programmatic 
change  that  might  also  have  affected  women's 
decisions  to  enroll  in  WIC. 

Process  or  Descriptive  Studies 

Market  Research  Study,  WIC  Program,  Oregon  State 
Health  Division 

Griggs- Anderson  (market  research  firm) 

Citation:  WIC  Program,  Oregon  State  Health  Division, 
P.O.  Box  231,  Portland,  Oregon  97207 

Research  Focus:  Market  research  study  to  identify  ways 
to  modify  WIC  services  and  planning  strategies  to 
improve  program  effectiveness 


Population  Studied:  WIC  participants  and  local  agency 
staff  in  Oregon 

Research  Objectives:  The  research  was  designed  to  meet 
five  objectives: 

■  Collect  information  to  improve  program 
effectiveness; 

■  Examine  WIC  services  as  perceived  by  staff  and 
clients; 

■  Profile  needs,  suggestions,  and  attitudes  of 
participants  and  staff; 

■  Analyze  findings  and  develop 
recommendations  and  guidelines  for  future 
planning;  and 

■  Communicate  findings. 

Study  Design:  The  study  included  both  qualitative 
and  quantitative  components.  For  the  qualitative 
component,  Griggs-Anderson  conducted  indepth, 
inperson  interviews  with  10  WIC  staff  in  four  different 
counties  and  39  WIC  participants  in  six  different 
counties.  The  counties  and  participants  represented 
a  mix  of  program  types  and  clients.  Interviews  lasted 
1  hour  and  45  minutes.  Interview  guides  were  used. 
Using  information  from  the  qualitative  component 
of  the  study,  Griggs- Anderson  developed  survey 
instruments  for  the  quantitative  analysis  component 
which  involved  telephone  interviews  with  127 
WIC  participants  and  50  local  agency  staff  persons. 
Participants  were  randomly  selected  and  represented 
all  30  counties  in  Oregon.  Staff  were  also  randomly 
selected,  though  quotas  were  set  to  include  all  staff 
positions. 

Analytic  Methods:  The  quantitative  analysis  consisted 
of  descriptive  statistics-rating  scales  and  frequencies. 
In  addition  to  informing  questionnaire  design,  the 
qualitative  data  were  reviewed  and  used  to  interpret 
the  quantitative  findings. 

Study  Findings:  The  results  of  this  study  are  divided 
into  six  categories  in  the  published  report.  An 
overview  of  these  findings  includes: 
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General  characteristics.  While  both  staff  and 
participants  said  there  is  no  "typical  WIC  client," 
the  study  found  that  most  (74  percent)  Oregon 
WIC  participants  are  married  with  an  average 
of  two  children  per  family.  Most  WIC  staff  have 
several  areas  of  responsibility. 

Awareness  of  WIC  services.  Most  participants 
understand  the  program's  purpose  and  the  services 
offered. 

Participant  satisfaction.  Participants  give  the  program 
an  overall  rating  of  8  on  a  scale  of  1  to  10,  with  10 
being  the  highest  rating.  They  value  other  WIC 
services,  particularly  the  emotional  support  from 
WIC  staff,  as  much  as  the  free  food.  Some  WIC 
cUents  find  monthly  food  instrument  pickup  to  be 
inconvenient  but  report  that  relationships  with 
"good  staff"  can  influence  regular  food  instrument 
pickup. 

Staff  rate  their  success  in  effecting  change  in  client 
behavior  from  25  percent  to  98  percent.  Staff  believe 
that  participants  primarily  come  to  the  WIC  clinic 
for  the  free  food  and  are  not  satisfied  with  WIC 
services.  They  regret  that  they  cannot  provide  more 
emotional  support  to  clients.  They  are  frustrated 
by  clients  who  do  not  show  up  for  appointments. 
Both  participants  and  staff  feel  that  young,  single 
mothers  need  the  most  help  but  are  often  the  most 
difficult  group  to  reach  and  to  educate. 

Perceptions  of  WIC.  Perceptions  of  WIC  are  generally 
positive.  Participants  value  the  food  they  receive, 
the  nutrition  education,  and  the  individual  time 
with  nutritional  staff.  One  weakness  identified  by 
participants  is  the  chaotic  and  rushed  atmosphere. 
Both  participants  and  staff  think  that  WIC  is  doing 
a  good  job  with  regard  to  "important"  factors — 
friendly  and  understanding  staff,  for  example. 
There  is,  however,  room  for  improvement  on  some 
of  the  less  important  factors-such  as  providing 
child  care  at  the  WIC  clinic.  The  ideal  WIC  Program 
would  offer  more  food,  "fun"  classes,  and  flexible 
appointment  times. 

Nutrition  education.  Participants  say  they  place  equal 
value  on  the  food  and  the  nutrition  education, 
though  participants  also  say  that  quality  nutrition 
education  "should  be"  practical,  creatively 
presented,  and  easy  to  implement.  They 


particularly  Uke  recipes  and  demonstrations.  Staff 
report  that  preparing  such  classes  is  very  "time 
consuming." 

Staff  perceptions  of  their  jobs.  Staff  are  generally 
satisfied  with  their  jobs — reporting  an  overall  rating 
of  8  on  a  1-to-lO  scale  with  10  as  the  highest  rating 
and  also  hold  their  coworkers  in  high  regard.  Local 
WIC  staff  though  are  discouraged  by  the  lack  of 
time  to  meet  all  needs,  the  hectic  environment,  and 
"negative"  clients. 

Recommendations  include: 

■  Make  client  satisfaction  the  priority 

■  Personalize  WIC's  approach  and  environment 

■  Market  WIC  as  a  fun  program  that  can  make  a 
family's  life  better 

■  Develop  curriculum  and  creative  techniques  at 
a  State  level  to  provide  support  to  local  staff 

M  Promote  staff  consistency  so  that  WIC 
"practices  what  it  preaches" 

■  Clarify  priority  cUent  populations 

Intended  Audience:  State  and  local  WIC  staff  in  Oregon, 
WIC  professionals  in  other  States. 

Applicahility:  This  type  of  study  could  be  conducted 
by  other  State  WIC  agencies  or  by  large  local  WIC 
agencies.  Because  this  study  was  undertaken  by  a 
professional  market  research  organization,  resource 
availability  is  a  major  issue.  The  published  report  does 
not  include  information  on  study  costs.  Copies  of  the 
interview  guide  and  the  survey  instrument  are  not 
included  in  the  published  report. 

The  research  appears  to  give  a  fairly  comprehensive 
picture  of  the  perceptions  of  both  WIC  participants 
and  staff  with  regard  to  WIC  program  effectiveness 
and  areas  for  improving  effectiveness. 
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Minnesota  WIC  Outreach  Plan  1987 

Spangler/Fisher  Advertising,  Inc. 

Citation:  Minnesota  Department  of  Health,  Division  of 
Maternal  and  Child  Health,  Minneapolis,  MN,  1987 

Research  Focus:  Identify  the  characteristics  of  current 
WIC  participants  and  of  eligible  nonparticipants. 
Develop  a  statev^ide  outreach  plan  to  market  WIC  to 
high-priority  nonusers. 

Population  Studied:  WIC  participants  and  eligible 
nonparticipants  in  Minnesota 

Research  Objectives:  Primary  objectives  of  this  study 
were  to  identify  and  describe: 

■  The  characteristics  of  WIC-eligible  women  who 
do  not  receive  WIC  benefits; 

■  The  best  methods  for  reaching  these  women; 

■  The  effectiveness  of  current  outreach  activities; 

■  Barriers  to  participation  in  the  WIC  Program; 

■  Differences  across  the  State  in  the  four 
objectives  above;  and 

■  The  appropriateness  of  a  role  for  the  State  WIC 
agency  in  outreach 

Study  Design:  The  research  period  covered  4  months 
beginning  in  the  fall  of  1986  and  encompassed  four 
types  of  data  collection:  indepth  interviews  with  20 
WIC  projects  across  the  State;  interviews  with  16 
referral  agencies  across  the  State;  a  survey  of  249 
clients  of  the  referral  agencies;  and  a  telephone  survey 
of  211  WIC  participants  and  273  low-income  women 
who  were  not  using  WIC. 

There  is  no  information  in  the  published  report  on 
sample  selection  of  WIC  agencies,  referral  agencies, 
referral  agency  participants,  WIC  participants,  or 
low-income  nonparticipants.  The  only  questionnaire 
included  in  the  report  appears  to  be  for  the  telephone 
survey  of  participants  and  nonparticipants.  It  can  be 
inferred  from  the  discussion  of  the  findings  that  the 
same  or  a  similar  instrument  was  used  to  survey  users 
of  referral  agency  services. 


Analytic  Methods:  Descriptive  analysis  of  the  survey 
data-response  frequencies  for  every  item  on  the 
telephone  survey  are  reported.  Extensive  examination 
of  the  more  qualitative  data  obtained  from  WIC  and 
referral  agencies. 

Study  Fi1^dings:  WIC  has  high  visibility  among 
low-income  women.  These  women  are  well  served 
by  current  outreach  efforts  which  rely  primarily  on 
word-of-mouth  (relatives,  friends,  and  neighbors). 
Most  (81  percent)  WIC  clients  know  someone  else  on 
WIC.  WIC  users  give  the  program  "exceptionally  high 
praise."  WIC  participants  have  higher  participation 
rates  (34  versus  10  percent)  in  the  AFDC,  Food  Stamp, 
and  Energy  Assistance  Programs  than  do 
nonparticipants. 

Women  with  family  incomes  in  the  $11,000  to  $20,000 
range  do  not  have  accurate  information  on  WIC 
services  or  on  eligibility  criteria  for  WIC.  They  view 
WIC  as  a  welfare  program.  These  women  are  not  part 
of  the  social  service  network,  so  more  and  different 
outreach  efforts  are  necessary  to  reach  them. 

Most  local  WIC  agencies  do  not  place  a  high  priority 
on  outreach — most  have  full  caseloads.  Further,  local 
staff  are  not  trained  in  outreach  activities,  do  not  know 
the  State's  objectives  with  regard  to  outreach,  and 
have  difficulty  producing  well-designed  outreach 
tools  (brochures  and  the  like). 

Referral  agencies  do  not  see  large  numbers  of  the 
higher  income  women.  In  addition,  some  of  these 
referring  agencies  do  not  have  sufficient  information 
on  WIC  eligibility  criteria. 

The  report  recommends  that  Minnesota  WIC  and 
its  local  service  providers  target  outreach  to  the 
higher-income  women  while  maintaining  the 
lower-income  caseload.  Some  other  recommendations 
include  expanded  contacts  with  key  referral  agencijs, 
outreach  training  for  local  WIC  staff,  and  regional 
outreach  planning.  The  final  section  of  the  report 
presents  a  series  of  specific  outreach  strategies  and 
activities  for  State  WIC  and  the  local  agencies. 

Intended  Audience:  State  and  local  WIC  staff  in 
Minnesota,  WIC  professionals  in  other  States 

Applicability:  If  a  State  WIC  agency  needs  information 
on  the  demographics  and  characteristics  of  its  eligible 


31 


non-WIC  population,  this  type  of  market  research 
would  be  useful.  The  report  presents  highly  specific 
outreach  planning  activities  and  strategies  that  could 
be  applied  in  other  WIC  agencies. 

Wisconsin  Standard  Deviations  Report 

Connie  Welch 

Citation:  Wisconsin  WIC  Program 

Research  Focus:  Comparative  statistics  for  local  WIC 
agencies 

Population  Studied:  All  local  WIC  agencies  in 
Wisconsin 

Research  Objective:  Summarizing  information  on  the 
Wisconsin  WIC  population.  Data  on  12  indicators 
are  drawn  from  59  local  agencies.  The  report  flags 
percentages  that  are  more  than  one  standard  deviation 
from  the  mean. 

Study  Design:  The  Wisconsin  Standard  Deviations 
Report  presents,  on  a  quarterly  basis,  by  local  agency, 
nutrition  and  administrative  information  about  WIC 
clients.  The  reported  data  include: 

■  Percent  of  infants  (born  to  prenatal  WIC  clients) 
breastfed  at  least  once  a  month  or  more; 

■  Percent  of  infants  who  do  not  receive  solid 
foods  until  they  are  4  months  of  age; 

■  Percent  of  infants  and  children  who  are 
certified  for  WIC  with  low  hematocrit  or 
hemoglobin  values  who  improve  (higher 
hematocrit  or  hemoglobin  values)  at 
recertification; 

■  Percent  of  low-birthweight  infants  born  to  WIC 
mothers; 


certification  period;  and  percentage  of  participants 
who  receive  at  least  one  health  contact  during  the 
certification  period; 

■  Percent  of  women  and  infants  enrolled  in  WIC; 

■  Percent  of  women  enrolled  in  WIC  during  the 
first  trimester; 

■  No-show  rate  (for  WIC  food  instrument 
pickup);  and 

■  Percent  of  food  instruments  issued  but  not 
cashed. 

Each  local  agency  receives  a  quarterly  report  on  that 
local  agency.  State  averages  (or  means),  and  the 
percentages  that  are  more  than  one  standard  deviation 
from  the  State  mean. 

Analytic  Methods:  Local  agency  data  are  downloaded 
from  the  automated  data  system  mainframe,  sent  to 
the  State  WIC  agency  on  diskettes,  and  placed  in 
a  spreadsheet  format  to  calculate  agency-specific 
frequencies  and  means  as  well  as  statewide 
frequencies  and  means. 

Study  Findings:  Not  applicable. 

Intended  Audience:  State  staff  use  composite  reports  to 
monitor  and  to  provide  technical  assistance  to  local 
agencies.  Local  agency  staff  compare  agency-specific 
data  with  statewide  information.  Local  agencies  may 
be  required  to  prepare  workplans  (nutrition  education 
plans,  for  example)  for  indicators  more  than  one 
standard  deviation  from  the  mean. 

Applicability:  The  data  in  the  report,  its  format, 
distribution,  and  use  could  be  replicated  by  other 
States. 


■  Percent  of  pregnant  women  who  report  that 
they  smoke; 

■  Percent  of  pregnant  women  who  report  that 
they  drink; 

■  Percent  of  participants  who  receive  at  least  one 
secondary  nutrition  contact  during  the 
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Appendix  B 
Sample  Instruments 


Record  Abstraction  Forms 

The  next  page  is  a  sample  of  one  type  of  record 
abstraction  form.  This  form  was  used  in  an 
FNS-funded  study  of  the  management  of  WIC  funds 
for  nutrition  and  program  administration.  It  was 
designed  to  allow  a  data  collector  to  review  up  to  10 
WIC  case  records.  If  information  on  each  topic  was 
recorded  in  the  case  record,  the  data  collector  placed 
a  check  in  the  appropriate  box.  Blank  boxes  indicated 
that  such  data  were  not  found  in  the  case  records.  The 
sample  includes  only  one  page  of  a  three-page  form. 
Data  collectors  can  also  use  a  record  abstraction  form 
to  copy  data  rather  than  indicating  the  presence  or 
absence  of  information.  B-2  is  a  sample  of  this  type  of 
record  abstraction.  This  sample  form  was  designed  for 
a  study  for  the  Centers  for  Disease  Control.  Data 
collectors  abstract  from  patient  records  the  dates  and 
results  of  certain  laboratory  tests. 

Interview  Guides 

Interview  guides  are  semistructured  instruments  that 
are  often  used  to  collect  data  on  program  organization 
and  operations.  They  offer  the  means  for  conducting 
more  informal  interviews,  but  they  provide  sufficient 
structure  so  that  the  same  topics  are  discussed  with 
each  respondent.  The  format  is  question-and-answer, 
but  the  questions  are  not  in  a  designated  order.  Also, 
interviewers  are  trained  to  use  respondents'  answers 
in  phrasing  foUowup  questions. 

On  B-3  are  several  questions  that  might  appear  on  an 
interview  guide  for  collecting  data  on  day-to-day 
operations  of  a  drug  abuse  information  and  referral 
service.  Please  note  that  this  example  is  only  the 
first  page  of  an  interview  guide.  It  is  by  no  means 
sufficiently  developed  to  collect  information  on  the 
design,  implementation,  and  operation  of  a  drvig 
abuse  information  and  referral  service  at  a  local  WIC 
agency. 

Survey  Instruments 

B-4  is  the  first  page  of  a  telephone  survey 
questionnaire.  This  questionnaire  was  also  used  in 
studying  the  management  of  WIC  funds  for  nutrition 
and  program  administration.  More  than  50  State 
WIC  directors  were  interviewed  during  this  survey. 


appear  in  one  style  while  instructions  to  the 
interviewer  on  procedures  and  appropriate  answers 
appear  in  another  style. 

A  one-page  sample  of  a  mail  questionnaire  appears  on 
B-5.  This  sample  is  drawn  from  the  1990  WIC 
participant  and  program  characteristics  study  (PC90). 
Staff  in  State  WIC  agencies  were  to  answer  these 
questions  and  then  return  the  forms  in  preaddressed, 
stamped  envelopes. 

Note  that  a  mail  questionnaire,  which  is  a 
self-completed  instrument,  must  contain  very  clear, 
quite  precise  instructions  so  that  comparable  data  are 
collected  across  all  respondents.  As  with  the  telephone 
survey  instrument,  different  typefaces  and  styles  (bold 
for  example)  are  used  to  call  respondent  attention  to 
the  various  instructions. 

Site-specific  or  respondent-specific  identifiers  do  not 
appear  on  these  one-page  samples.  The  assumption 
here  is  that  the  instrument  will  have  a  cover  page  on 
which  site-specific  or  respondent-specific  identifiers 
are  recorded. 


Observation  Forms 

Much  can  be  learned  by  walking  through  a  WIC 
agency  and  observing  the  waiting  areas  and  the  staff 
at  work  as  well  as  the  way  in  which  space  is  used 
by  both  WIC  participants  and  staff.  Observation 
checklists  can  be  informal-a  series  of  reminders  to 
observers  to  look  for  certain  items  or  situations. 
Or,  observation  checklists  can  pose  more  specific 
questions. 

The  sample  on  the  next  page  (B-6)  is  a  checklist  for 
collecting  information  on  nutrition  education  in  a  local 
WIC  agency.  The  form  is  designed  so  that  an  observer 
can  record  information  on  several  sites  operated  by  a 
local  WIC  Program.  This  checklist  was  used  in  the 
FNS-funded  study  of  the  management  of  WIC  funds 
for  nutrition  and  program  administration. 


Note  that  the  information  and  questions  that  the 
telephone  interviewer  will  read  to  the  respondent 
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B-1 

RECORD  REVIEW  FORM 


#1 

#2 

#3 

#4 

Cli( 
#5 

3nt 
#6 

#7 

#8 

#9 

#10 

Date 

(^HQnt  rXrrw  \t~\  ID    DD    RP    1  0\ 

'ylienT  LarOUp      ,  rr,  Dr,  1,  Kj) 

Anthropometrics 

Are  client's  height  and  weight  recorded? 

Are  data  plotted  on  a  growth  chart? 

Does  the  chart  include  data  on  the  client's: 
Weight  for  age 

Height  for  age 

Weight  for  height 

Head  circumference 

Is  data  entered  within  1/2  pound 
or  MZ  men  r 

Food  delivery 

Does  file  contain  information  on: 
Current  food  package 

Resolution  of  problems 

Referrals 

Has  client  been  referred  to: 
On-site  health  care 

Vw/Lllcl  bciVIOco  Ull  oILc 

Off-site  health  care 

Other  services  off  site 

Is  there  any  record  of  followup  of  referrals? 
SPECIFY 

Is  there  any  documentation  of  referrals? 
SPECIFY 

B-2 


Patient  ID   Review  Date 


For  each  test,  record  the  test  results  and  the  date  of  the  test,  if  these  data  are  in  the  records. 
If  a  test  has  been  repeated,  record  (1)  the  date  and  results  of  the  test  closest  to  onset  of 
this  illness,  and  (2)  the  date  and  results  of  the  test  nearest  to  the  present  date.  If  a  date  is 
not  in  the  records,  write  "NA"  on  the  line  for  the  date. 

If  the  local  laboratory  indicates  "normal  values"  for  any  test  result,  please  record  this  infor- 
mation next  to  the  name  of  the  test. 


DATE  RESULT      DATE  RESULT 

AUTOIMMUNE  DISEASES 

ANA         


RA 


Other  (Anti-thyroid,  Anti-DS-DNA,  Anti-smooth  muscle) 


COMPLETE  BLOOD  COUNT 
Hgb 
Hct 
WBC 
Platelets 
Differential  WBC 

%  Polys 

Bands 

Lymphs 

Atypical  lymphs 
Monos 
Eos 
Other 
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INTERVIEWERS:  Please  discuss  the  following  issues  and  topics  with  staff  involved  in  designing, 
implementing,  and/or  providing  drug  abuse  information  and  referral  services  in  this  WIC  agency. 
The  interview  need  not  be  limited  to  the  issues  listed  on  this  interview  guide.  Remember  to  take 
notes  on  any  other  issues  discussed. 


1.      Program  History 

What  were  the  factors  that  most  influenced  how  this  drug  abuse  information  and  referral 
service  was  designed? 

Did  staff  or  anyone  else  feel  a  need  for  such  a  service  before  WIC  regulations  required  it? 

Who  was  involved  in  designing  the  drug  abuse  information  and  referral  service?  Were  the 
designers  all  WIC  staff  or  were  others  involved? 

Who  implemented  it?  What  problems  were  encountered  during  implementation? 

How  were  they  resolved?  Have  they  been  resolved?  If  not,  why  not? 

What  was  the  timeframe  for  designing  the  program?  for  implementing  it? 

Was  there  formal  staff  training?  Who  designed  the  training  effort?  Who  actually  provided 
the  training?  What  was  the  duration  of  staff  training?  What  types  of  staff  were  trained?  Did 
all  staff  receive  the  same  training? 


2.      Program  Operations 

Who  provides  drug  abuse  information  and  referral  services  to  WIC  participants? 
Where  are  these  services  provided? 
How  often  are  such  contacts  made? 

Are  such  contacts  separately  scheduled?  If  so,  what  is  the  scheduling  procedure? 

What  types  of  media  are  used?  Where  and  how  are  these  materials  obtained? 

Are  specific  drugs  discussed  or  is  the  approach  more  general?  What  are  the  reasons  for 
the  chosen  approach? 

Are  there  formal  referral  procedures  for  participants  needing  more  than  information  on  the 
potential  problems  that  drug  abuse  can  create?  If  yes,  what  are  they?  If  no,  how  are  such 
referrals  handled? 


B-4 


Because  there  are  so  many  differences  between  and  among  WIC  agencies,  we  would  like  to  begin 
this  interview  by  asking  you  some  questions  that  will  help  us  to  characterize  and  describe  your 
WIC  program. 

1 .  When  did  you  begin  managing  this  WIC  program? 

 /  

MONTH  YEAR 

2.  When  did  you  begin  working  for  this  WIC  program? 

 /  

MONTH  YEAR 


3.  Did  you  work  for  any  other  WIC  program  before  joining  the  staff  of  this  program? 

Yes  ASK  QUESTION  3A.  1 

No  SKIP  TO  QUESTION  4.  2 

3A.  How  long  did  you  work  for  WIC  before  coming  to  this  program? 


MONTHS 

INTERVIEWER:      DOUBLE  CHECK  RESPONDENT'S  ANSWER  TO  CIJ\RIFY 
WHETHER  S/HE  SPECIFIES  YEARS  OR  MONTHS.  CONVERT  ALIJ\NSWERS  TO 
MONTHS. 


4  .  For  fiscal  year  1968,  what  is  your  total  WIC  administrative  budget? 

$  

SPECIFY  WHOLE  DOLLARS. 


5  .  Of  your  total  1986  administrative  budget,  what  portion  of  the  budget  is  spent  at  the  State  level? 

$  

SPECIFY  WHOLE  DOLLARS. 

or 

 % 

PERCENT  OF  BUDGET 


B-5 


SUBMITTED  BY 


Name  of  State  WIC  Agency 


Income  Determination 


A1.  Describe  the  State  Agency  (SA)  gross  income  limit  for  eligibility.  CIRCLE  ONE  NUMBER. 

SA  uses  the  standard  1 85  percent  of  poverty  guidelines 

published  in  the  Federal  Register  1 

SA  uses  other  standards  DESCRIBE  BELOW  2 

Economic  Unit                     Income  Limit  or  Percent  of  Poverty 

(gross  per  annum) 

1     

2     

3     

4     

5     

6     

7     

8     

9     


A2.  What  programs  and/or  respective  income  limits  are  used  by  the  State  Agency  for  adjunctive 
eligibility  determinations? 

CIRCLE  THE  PROGRAMS  AND  SPECIFY  THE 
MAXIMUM  PERCENT  OF  POVERTY  ALLOWED 
FOR  THOSE  PROGRAMS. 


Program                                                         Percent  of  Poverty 
AFCD   1   


Food  Stamp  Program   2   130% 

Medicaid   3   


551   4. 


Reduced-Price  School  Lunch   5   185% 

Free  School  Lunch   6   130% 

Other  programs  SPECIFY   7   
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LOCAL  AGENCY. 


Nutrition  Education 

Record  a  Y  for  Yes,  an  N  for  No,  or  an  NA  for  Not  Available.  Record  site  names  at  the  top  of 
each  column. 


Topic 

Site 

Site 

Site 

NUTRITION  EDUCATION  SESSIONS 
Is  nutrition  education  conducted 
on  site? 

Is  nutrition  education  conducted 
during  .  . 

certification? 

food  instrument  pickup? 

waiting  periods? 

dietary  assessment? 

Are  nutrition  education  sessions 
scheduled  independently  of 
other  activities? 

MATERIALS 

Are  the  following  nutrition 
education  materials  available  in 
the  waiting  area  

pamphlets/brochures 

posters 

recipes 

audiovisuals 
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Appendix  C 

USDA  Resources  for  Planning 
and  Evaluation 


The  following  are  USDA  resources  available  to  State 
and  local  WIC  staff  interested  in  evaluation: 

1.  The  National  Agricultural  Library  Food  and 
.    Nutrition  Information  Center  (FNIC)  -  FNS 

funds  this  information  center  to  make  library 
services  available  to  you  on  WIC  and  related 
subjects.  FNIC  can  provide  you  with  literature 
searches  on  WIC  research  including  FNS 
reports  and  major  journals  that  publish  WIC 
research,  bibliographies  on  special  topics,  and 
copies  of  research  articles.  The  public  can  also 
borrow  materials  from  the  library.  Contact: 
301-344-3719. 

2.  FNS  Reports  and  Materials  -  National  and 
Regional  FNS  staff  may  be  aware  of  data 
sources  or  evaluations  of  certain  aspects  of 
WIC  and  oversee  national  studies  of  the 
program.  Contact  your  regional  office  or 
the  Supplemental  Food  Program  Division 
(703-756-3746)  or  the  Office  of  Analysis 
and  Evaluation  (703-756-3115)  at  FNS 
Headquarters. 


3.  USDA/ Agricultural  Research  Service  Human 
Nutrition  Centers  -  There  are  5  USDA  Human 
Nutrition  Centers,  two  may  be  of  interest  to 
WIC  staff: 

■  the  Child  Nutrition  Research  Center  at  Baylor 
College  of  Medicine  is  the  only  research  center 
that  deals  exclusively  with  research  on  nutrient 
needs  and  nutritional  status  of  mothers,  infants, 
and  children.  Contact:  713-798-7018. 

■  the  Western  Human  Nutrition  Research 
Center's  mission  is  to  improve  methods  for 
assessing  human  nutritional  status  and  to  study 
the  factors  that  lead  to  malnutrition.  Contact: 
415-556-9699. 
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